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Preface 


In the period February 5-7, 2007 we organized in the historie building of 
the Dutch Royal Academy of Sciences (KN AW) in Amsterdam the Academy 
Colloquium titled "New Perspectives on Games and Interaction". The pro- 
gram consisted of 14 invited lectures, each followed by a commentary, and 8 
contributed talks. 

Traditionally, logic and linguistics have been studied from a statie and 
non-interactive point of view emphasizing the structure of proofs and mean- 
ings. In computer science, dynamic processing of information has always 
played a major role, but from a non-interactive machine perspective. More 
recently, the dynamic and interactive aspects of logical reasoning, commu- 
nication, and information processing have been much more central in the 
three above-mentioned disciplines. 

Interaction is also of crucial importance in economics. Mathematical 
game theory, as launched by Von Neumarm and Morgenstern in 1944 in 
their seminal book, followed by the contributions of Nash, has become a 
standard tooi in economics for the study and description of various economie 
processes, including competition, cooperation, collusion, strategie behaviour 
and bargaining. 

These different uses of games in logic, computer science, linguistics and 
economics have largely developed in isolation. The purpose of the workshop 
was to bring together the researchers in these areas to encourage interactions 
between these disciplines, to clarify their uses of the concepts of game theory 
and to identify promising new directions. 

This volume consists of the contributions written by the speakers. It 
testifies to the growing importance of game theory as a tooi to capture the 
concepts of strategy, interaction, argumentation, communication, cooper- 
ation and competition. We hope that the reader will find in the papers 
presented in this volume useful evidence for the richness of game theory 
and for its impressive and growing scope of use. 

We take this opportunity to thank Benedikt Lowe and Johan van Ben- 
them for their cooperation in the preparations of the scientific programme 
of the Colloquium. 


Amsterdam K.R.A. R.A.M.v.R. 
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Abstract 


We present a logic of conditional doxastic actions, obtained by in- 
corporating ideas from belief revision theory into the usual dynamic 
logic of epistemic actions. We do this by reformulating the setting 
of Action-Priority Update (see Baltag and Smets, 2008) in terrns of 
eonditional doxastie models, and using this to derive general reduction 
laws for conditional beliefs after arbitrary actions. 


1 Introduction 


This work is part of the on-going trend (see Aueher, 2003; van Benthem, 
2004; van Ditmarseh, 2005; Baltag and Sadrzadeh, 2006; Baltag and Smets, 
2006a,b,e, 2007a,b, 2008) towards ineorporating belief revision meehanisms 
within the Dynamie-Epistemie Logie (DEL) approach to information up- 
date. As sueh, this paper ean be eonsidered a sequel to our recent work 
(Baltag and Smets, 2008), and it is based on a revised and improved ver- 
sion of our older unpublished paper (Baltag and Smets, 2006e), presented 
at the 2006 ESSLLI Workshop on "Rationality and Knowledge". 

We assume the general distinetion, made by van Ditmarseh (2005), Bal- 
tag and Smets (2006a) and van Benthem (2004), between "dynamie" and 
"statie" belief revision. In this sense, classieal AGM theory in (Alehourré6n 
et al., 1985) and (Gärdenfors, 1988) (and embodied in our setting by the eon- 
ditional belief operators BEQ) is "statie", eapturing the aqent's ehanging 
beliefs about an unehanging uiorld. As sueh, "statie" belief revision eannot 
be self-referential: statieally-revised beliefs eannot refer to themselves, but 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 9-31. 
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only to the original, unrevised beliefs. Incontrast, "dynamic" belief revision 
deals with the agent's revised beliefs about the world as it is afier revision 
(including the revised beliefs themselves). 

In (Baltag and Smets, 2006a), we proposed two equivalent semantie set- 
tings for "statie" belief revision (eonditional doxastie models and epistemie 
plausibility modeis), and proved them to be equivalent with eaeh other and 
with a multi-agent epistemie version of the AGM belief revision theory. We 
argued that these settings provided the "right" qualitative semanties for 
multi-agent belief revision, forrning the basis of a eonditional doxastie logie 
(CDL, for short), that eaptured the main "laws" of statie belief revision us- 
ing eonditional-belief operators BË Q and knowledge operators KaP. The 
later correspond to the standard S5-notion of "knowiedge" (partition-based 
and fuliy introspeetive), that is eommonly used in Computer Seienee and 
Game Theory. In the same paper, we went beyond statie revision, using 
CDL to explore a restrieted notion of "dynamic" belief revision, by mod- 
eling and axiomatizing multi-agent belief updates indueed by publie and 
private announeements. 

In subsequent papers, eulminating in (Baltag and Smets, 2008), we ex- 
tended this logie with a "safe belief’ modality DaP, eapturing a form of 
"weak (non-introspeetive) knowiedge" , first introdueed by Stalnaker in his 
modal formalization (Stalnaker, 1996, 2006) of Lehrer's defeasibility anal- 
ysis of knowledge (Lehrer, 1990; Lehrer and Paxson, 1969). We went on 
to deal with "dynamic" multi-agent belief revision, by developing a no- 
tion of doxastie actions", general enough to cover most examples of muiti- 
agent eommunieation actions eneountered in the literature, but also ftexible 
enough Lo implement various "belief-revision policies" in a unified setting. 
Foliowing Aueher (2003) and van Ditmarseh (2005), we represented dox- 
astie actions using (epistemic) action plausibility models. The underlying 
idea, originating in (Baltag and Moss, 2004) and (Baltag et al., 1998), was 
to use the same type of formalism that was used to model "statie" beliefs: 
epistemiejdoxastie actions should be modeled in essentially the same way as 
epistemiejdoxastie states. The main differenee between our proposal and 
the proposals of Aueher (2003), van Ditmarseh (2005) and Baltag et al. 
(1998) lies in our different notion of "update product" of a state model with 
an action model: our "Aetion-Priority Update" was based on taking the 
anti-lexieogmphie order on the Cartesian product of the state model with 
the action model. This is a naturel genemlization of the AGM-type belief 
revision to complex multi-agent belief-ehanging aetions: foliowing the AGM 
tradition, it gives priority to ineoming information (i.e., to "aetions" in our 
sense). Inthe same paper (Baltag and Smets, 2008), we eompletely axiom- 
atized the generallogie of dynamie belief revision, using Reduetion Axioms 


1 Or "doxastic events", in the terminology of van Benthem (2004). 
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for knowledge and safe belief aftel' arbitrary doxastic actions. 

In this paper, we go further to look at representations of doxastic ac- 
tions in terrns of our other (equivalent) semantic setting for belief revision 
mentioned above (conditional doxastic modeis). We look in detail at an 
equivalent statement for the (same) notion of Action-Priority Update in 
terrns of conditional doxastic actions. This is in itself a non-trivial, rather 
intricate exercise, which as a side benefit gives us Reduction Axioms for 
conditional belief aftel' arbitrary actions. Though more complex than the 
Reduction Axioms for knowledge and safe beliefin (Baltag and Smets, 2008) 
(and in principle derivable from these"), the axioms of the resulting Logic of 
Conditional Doxastic Actions are of more direct relevanee to belief revision 
and belief update, and are immediately applicable to deriving reduction laws 
for interesting special cases, such as the ones considered by van Benthem 
(2004). 

In its spirit, our approach is closer to the one taken by J. van Benthem 
and his collaborators (van Benthem, 2004; van Benthem and Roy, 2005; 
van Benthem and Liu, 2004) (based on qualitative logies of eonditional be- 
lief, "preference" modalities and oarious [orms of "belief upgrade”), rather 
than to the approaches of a more "quantitative" ftavor due to Aucher (2003) 
and van Ditmarsch (2005) (based on formalizing Spohn's ordinal degrees of 
beliefs (1988) as "graded belief’ operators, and proposing various quantita- 
tive arithmetic formulas for updates). As aresuit, the "reduction axioms" 
by van Benthem (2004) (for "hard" public announcements, lexicographic 
upgrades and conservative upgrades) can be recovered as special cases of 
our main reduction axiom for conditional beliefs aftel' an arbitrary action. 

Our conditional belief modalit.ies and our condit.ional doxast.ic rnodels 
can also be seen in the context of the wide logical-philosophical literature 
on notions of eonditional (see, e.g., Adams, 1965; Stalnaker, 1968; Ram- 
sey, 1931; Lewis, 1973; Bennett, 2003). One can of course look at condi- 
tional belief operators as non-classical (and non-monotonic!) implications. 
Our approach can thus be compared with other attempts of using doxastic 
conditionals to deal with belief revision, (see, e.g., Gärdenfors, 1986; Ram- 
sey, 1931; Grove, 1988; Rott, 1989; Fuhrmann and Levi, 1994; Ryan and 
Schobbens, 1997; Halpern, 2003; Friedmann and Halpern, 1994). As shown 
in (Baltag and Smets, 2006a), our operators avoid the known paradoxes" 
arising from such mixtures of conditional and belief revision, by failing to 
satisfy the so-ealled Ramsey test. 


N 


Together with the axioms of the logic of knowledge and safe belief, and with the 
definition in (Baltag and Smets, 2008) of conditional belief in terms of knowledge and 
safe belief. 

See e.g., (Stalnaker, 1968), (Gärdenfors, 1988) and (Rott, 1989). 


w 
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2 Preliminaries: Epistemie plausibility models and 
conditional doxastic models 


In this section, we review some basic notions and results from (Baltag and 
Smets,2UU6a). 


Plausibility frames. An epistemic plausibility frame is a structure S 

(S, <«, Salaca, consisting of a set S, endowed with a family of equivalence 
relations <<, called epistemic indistinguishability relaiions, and a family of 
"well-preorders" <,, called plausibility relations. Here, a "well-preorder" is 
just a preorder" such that every non-empty subset has minimal elements.’ 
Using the notation Ming T := {t ET: t < ti for all ti E T} for the set 
of minimal elements of T, the last condition says that: for every T C S, if 


T #0 then Ming TF 0. 


Plausibility frames for only one agent and without the epistemic relations 
have been used as models for conditionals and belief revision by Grove 
(1988), Gärdenfors (1986, 1988), Segerberg (1998), etc. Observe that the 
conditions on the preorder <, are (equivalent to) Grove's conditions for the 
(relational version of) his models (Grove, 1988). The standard formulation 
of Grove models (in terms of a "system of spheres", weakening the similar 
notion in (Lewis, 1973)) was proved by Grove (1988) to be equivalent to the 
above relational forrnulation." 

Given a plausibility frame S, an S-pmposition is any subset P C S. We 
say that the state s satisfies the pmposition P if s E P. Observe that a 
plausibility frame is just a special case of a Kripke frame. So, as is standard 
for Kripke frames, we can define an epistemic plausibility model to be an 
epistemic plausibility frame S together with a valuation map 1-1: ® > 
P(S), mapping every element of a given set ® of "atomie sentences" into 
S-propositions. 


Notation: strict plausibility, doxastic indistinguishability. As with 
any preorder, the ("non-strict") plausibility relation <, above has a "strict" 
i.e., asymmetrie) version <a, as weil as a cortesponding equivalence relat ion 
“aq, called "doxastic indistinguishability”: 


s <a t iff s <a tand t Za s 
s œa t iff s <a tand t <a s 


4 Le., a reftexive and transitive relation. 

5 Observe that the existence of minimal elements implies, by itself, that the relation <, 
is both reflexive (i.e., s Sa s for all s E S) and connected (i.e., either s <q tort <a S, 
for all s, tÆ Su, i.e., elements that are below all the others. Note also that, when the 
set Sisfinite, a well-preorder is nothing but a connected preorder. 

6 A more concrete example ofplausibility frames was given by Spohn (1988), in terms of 
ordinal plausibility maps assigning ordinals d(S) ("the degree of plausibility" of s) to 
each state s E S. In our epistemic multi-agent context, this would endow each agent 
a with an ordinal plausibility map d.; : S — Ord. 
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Interpretation. The elements of S will be interpreted as the possible states 
of a system (or "possible worlds"). The atomie sentences p E ẸP represent 
"oniic" (non-doxastic) facts about the world, that might hold or not in a 
given state, while the valuation tells us which facts hold at which worlds. 
The equivalence relations ~ capture the agerii's knowledge about the actual 
state of the system (intuitively based on the agent's {pariial] observaiions of 
this state): two states s, t are indistinguishable fOT agent a ifs ~a t. Inother 
words, when the actual state of the system is s, then agent a knows only the 
state's equivalence class s(a) := {t ES: s ~a t}. Finally, the plausibility 
relations <, capture the agent's conditional beliefs about (virtual) states of 
the system: given the information that some possible state of the system is 
either s or t, agent a will believe the state to be s iff s <a t; will believe the 
state to be ¢ ifft <a s, otherwise (if s ~, t), the agent will consider the two 
alternatives as equally plausible. 


Example 1. The father informs the two children (Alice and Bob) that he 
has put a coin Iying face up on the table in front of them. At first, the face 
is covered (so the children cannot see it). Based on previous experience, (it 
is common knowledge that) the children believe that the upper face is (very 
likely to be) Heads: say, they know that the father has astrong preferenee 
for Heads. And in fact, they're right: the coin lies Heads up. Next, the 
fat her shows the face of the coin to Alice, in the presence of Bob but in such 
a way that Bob cannot see the face (though of course he can see that Alice 
sees the face). The plausibility model S for this situation is: 


ae ed 
J 


Here, we left the father out of the picture (sinee he only plays the role of 
God or Nature, not the role of an uncertain agent). The node on the left, 
labeled with H, represents the actual state of the system (in which the coin 
lies Heads up), while the node on the right represents the other possible state 
(in which the coin is Tails up). We use continuous arrows to encode Alice's 
beliefs and use continuous squares to eneode her knowiedge, while using 
dashed arrows and dashed squares for Bob. More precisely: the squares 
represent the agents' information cells, i.e., the equivalence classes s(a) := 
{t ES: s ~, t} of indistinguishable states (for each agent a). Observe that 
Alice's information cells (the continuous squares) are singletons: in every 
case, she knows the state of the system; Bob's information cell is one big 
dashed square comprising both states: he doesn't know which state is the 
realone, so he cannot distinguish between them. The arrows represent the 
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plausibility relaiions for the two agents; since these are always refiexive, we 
choose to skip all the loops for convenience. Both arrows point to the node 
on the left: a priori (i.e., before making any observation of the real state), 
both agents believe that it is likely that the coin lies Heads up. 


Conditional doxastic frames. A plausibility frame is in fact nothing 
but a way to encode al! the agents’ possible conditional beliefs. To see 
this, consider the following equivalent notion, introduced in (Baltag and 
Smets, 2üü6a): A conditional doxastic frame (CD-frame, [or short} S = 
(S, {ef }aca,p CS) consists of a set of states S, together with a family of 
conditional (doxastic) eppearanee maps, one for each agent a and each pos- 
sible condition P C S. These are required to satisfy the following conditions: 


1. if SE P then s? + O; 

2. if PNs? £0 then sË 40; 

3. ift e s? then s? = tẸ; 

4. s? <P; 

5. Ge. sP nQ, ifs? NOZO. 


A conditional doxastic model (CDM, for short) is a Kripke model whose 
underlying frame is a CD-frame. The conditional appearance s£ captures 
the way a state sappears to an agent a, given some additional (plausible, 
but not necessarilsj tTUthful) information P. More precisely: whenever s is 
the current state of the world, then aftel' receiving new information P, agent 
a wil! come to believe that any of the states si E s? might have been the 
cutrent state of the world (as it was before receiving information P). 

Using conditional doxastic appearance, the knowledge s(a) possessed by 
agent a about state s (i.e., the epistemic eppearanee of s) can be defined as 
the union ofall conditional doxastic appeamnces. In other words, something 
is known ijJ it is believed in any conditions: s(a) := UQc;s s2. Using this, 
we can see that the first condition above in the definition of conditional 
doxastic frames captures the truthfulness of knowiedge. Condition 2 states 
the success of belief reoision, when consistent with knowiedge: if something 
is not known to be false, then it can be consistently eniertained as a hypoth- 
esis. Condition 3 expresses full introspeetion of (conditional) beliefs: agents 
know their own conditional beliefs, so they cannot revise their beliefs about 
them. Condition 4 says hypotheses are hypothetically believed: when mak- 
ing a hypothesis, that hypothesis is taken to be true. Condition 5 describes 
minimality of revision: when faced with new information Q, agents keep as 
much as possible of their previous (conditional) beliefs a 
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To recover the usual, unconditional beliefs, we put sa := s$ In other 
words: unconditional ("default") beliefs ar-e beliefs conditionalized by tr-iv- 
ially irue conditions. 

For any agent a and any S-proposition P, we can define a conditional 
belief operator BP : P(S) — P(S) S-propositions, as the Galois dual of 
conditional doxastic appearance: 


BEQ = {sE S: CQ} 


We read this as saying that agenta believes Q given P. More precisely, this 
says that: ifthe agent would learn P, then (ajter- leaming) he would come 
to believe that Q was the case in the curreni state [bejore the leaming). The 
usual (unconditional) belief operator can be obtained as a special case, by 
conditionalizing with the trivially true proposition S: BaQ := BEQ. The 
knowledge operator can similarly be defined as the Galois dual of epistemic 
appearance: 
KaP:= {s ES: s(a) < P}. 


As a consequence of the above postulates, we have the following: 


K.p= N BQP = Bg’ @= By’ P 
Qcs 


Equivalence between plausibility models and conditional doxastic 
modeis. Any plausibility model gives rise to a CDM, in a canonical way, 
by putting 

sf te Ming, {t EP: t m s} 


where Min., T:= {t ET: t <, ti for all ti ET} is the set of <,-minimal 
elements in T. We call this the canonical CDM associated to the plausibility 
mode!. The converse is given by a: 


Theorem 2.1 (Representation Theorem). Every CDM is the canonieal 
CDM of some plausibility model." 


The advantage of the CDM formulation is that it leads naturally to 
a complete axiomatization of a logic of conditional beliefs, which was in- 
troduced in (Baltag and Smets, 2UU6a) under the name of "Conditional 
Doxastic Logic" (CDL)8: the semantical postulates that define CDM's can 
be immediately converted into modal axioms governing conditional belief. 


7 This result can be seen as an analogue in our semantic context of Gi.rdenfors' rep- 
resentation theorem (Gia.rdenfors, 1986), representing the AGM revision operator in 
terrns of the minimal valuat.ions for some total preorder on valuations. 

8 COL is an extension of the well-known logic KL of "knowledge and belief’; see e.g., 
(Meyer and van der Hoek, 1995, p. 94), for a complete proof system for KL. 
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Conditional Doxastic Logic (CDL). The syntax of CDL (without com- 
mon knowledge and common belief operators)? is: 


p := p| | gAg | Ble 


while the semantics is given by the obvious compositional clauses for the 
interpretation map ll-ilS : CDL — P(S) in a CDM (and so, in particular, 
in a plausibility model) S. In this logic, the knowledge modality can be 
defined as an abbreviation, putting Kap := By’. (where L= pA 'p 
is an inconsistent sentence), or equivalently Kay := Bfp. This way of 
defining knowledge in terms of doxastic conditionals can be traced back to 
Stalnaker (1968). It is easy to see that this agrees semantically with the 
previous definition of the semantic knowledge operator (as the Galois dual 
of epistemic appearance): ||Kayl|g = Kallcplls. 


Doxastic propositions. A doxastic pmposition is a map P assigning to 
each plausibility model (or conditional doxastic model) S some S-proposi- 
tion, i.e., a set of states PS C S. The interpretation map for the logic CDL 
can thus be thought of as associating to each sentence y of CDL a doxastic 
proposition lIcpll. We denote by Prop the family of all doxastic propositions. 
All the above operators (Boolean operators as weil as doxastic and epistemic 
modalities) on S-propositions induce corresponding operators on doxastic 
propositions, defined pointwise: e.g., for any doxastic proposition P, one can 
define the proposition KaP, by putting (KaP)s = KaPs, for all models S. 


Theorem 2.2 (Baltag and Smets 2006a). A complete proof system for 
CDL can be obtained from any complete axiomatization of propositional 
logic by adding the following: 


Necessitation Rule: From F ip infer H BY ip 
Normality: + Bly — p) > (B? p — Bev) 
Truthfulness of Knowiedge: F Kaip > ip 

Persistenee of Knowiedge: F Kap > BY ip 

Pull Introspection: H Bo > KaB% 


z ~B =R KBE 
Hypotheses are (hypothetically) 
accepted: + BY ip 

Minimality of revision: F =B? ay — (B^ 9 = BL (Yy > 9)) 


Proo]. The proof is essentially the same as of (Board, 2002). It is easy to 
see that the proof system above is equivalent to Board's strongest logic of 
(Board, 2002) (the one that includes axiom for full introspection), and that 
our models are equivalent to the "full introspective" version of the semantics 
of (Board, 2002). Q.E.D. 


9 In (Baltag and Smets, 2006a), we present and axiomatize a logic that includes condi- 
tional common knowledge and conditional common true belief. 


The Logic of Conditional Doxastic Actions 17 


3 Action plausibility models and product update 


The belief revision encoded in the models above is of a static, purely hypo- 
thetical, nature. Indeed, the revision operators cannot alter the models in 
any way: all the possibilities are already there, so both the unconditional 
and the revised, conditional beliefs refer to the same uiorld and the same 
moment in time. In contrast, a beliefupdate in our sense is adynamic form 
of belief revision, meant to capture the actual change of beliefs induced by 
learning (or by other forms of epistemicjdoxastic actions).10 As already no- 
ticed before, by e.g., Gerbrandy (1999) and Baltag et al. (1998), the original 
model does not usually include enough states to capture all the epistemic 
possibilities that arise in this way. So we now introduce "revisions" that 
change the original plausibility model. 

To do this, we adapt an idea coming from (Baltag et al., 1998) and 
developed in full formal detail in (Baltag and Moss, 2004). There, the 
idea was that epistemic actions should be modeled in essentially the same 
way as epistemic states, and this common setting was taken to be given by 
epistemic Kripke models. Since we now enriched our models for states to 
deal with conditional beliefs, it is natural to follow (Baltag and Moss, 2004) 
into extending the similarity between actions and states to this conditional 
setting, thus obtaining action plausibility models. 

An action plausibility model is just an epistemic plausibility frame & = 
(È, <<, Salaca, together with a precorulition map pre : © — FTOp associ- 
ating to each element of } some doxastic proposition pre(a). As in (Baltag 
and Moss, 2004), we call the elements of }X (basic) epistemic actions, and 
we call pre(a) the precorulition of action o. 


Interpretation: Beliefs about changes encode changes of beliefs. 
The name "doxastic actions" might be a bit misleading; the elements of a 
plausibility model are not intended to represent "reai" actions in all their 
complexity, but only the doxastic changes induced by these actions: each 
of the nodes of the graph represents a specific kind of change of beliefs (of 
all the agents). As in (Baltag and Moss, 2004), we only deal here with 
pure "belief changes", i.e., actions that do not change the "ontic" facts of 
the world, but only the agents' beliefs.1' iVIoreover, we think of these as 
deierminisiic changes: there is at most one output of applying an action to 
a state. !? Intuitively, the precondition defines the domain of applicability of 


10 But observe the difference between our notion of belief update (originating in dynamic- 
epistemic logic) and the similar (and vaguer) notion in (Katsuno and Mendelzon, 1992). 

11 We stress this is a minor restrietion, and it is very easy to extend this setting to 
"ontic" actions. The only reasen we stick with this restrietion is that it simplifies the 
definitions, and that it is general enough to apply to all the actions we are interested 
here, and in particular to all communication actions. 

12 As in (Baltag and Moss, 2004), we will be able to represent non-deterministic actions 
as sums (unions) of deterministic ones. 
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o: this action can be executed on a state s iff s satisfies its precondition. The 
plausibility pre-orderings <, give the agent's beliefs about which actions are 
more plausible than others. But this should be interpreted as beliefs about 
changes, that ericode changes of beliefs. In this sense, we use such "beliefs 
about actions" as a way to represent doxastic changes: the information 
about how the agent changes her beliefs is captured by our action plausibility 
relations. So we read o <a o’ as saying that: if agent a is given the 
information that some (virtual) action is either ø or o’ (without being able 
to know which), then she believes that ø is the one actually happening. 


Example 2: Successful lying. The action of "public successful lying" 
can be described as follows: given a doxastic proposition P, the model con- 
sists of two actions Lie, Pand True., P, the first being the action in which 
agent a publicly lies that (she knows) P (while in fact she doesn't know 
it), and the second being the action in which a makes a truthful public 
announcement that (she knows) P. The preconditions are pretl.ie, P) = 
aK,P and pre/'Irue, P) = KaP. Agent a's equivalence relation is simply 
the identity: she knows whether she's lying or not. The other agents' equiv- 
alenee relation is the total relaiion: they cannot know if a is lying or not. 
Let us assume that a's plausibility preorder is also the total relaiion: this 
would express the fact that agent a is noi decided to always tie; a priori, 
she considers equally plausible that, in any arbitrarily given situation, she 
will lie or not. But the plausibility relations should refiect the fact that we 
are modeling a "typically successful lying": by default, in such an action, 
the hearer trusts the speaker, so he is inclined to believe the lie. Hence, the 
relation for any hearer bÆ a should make it more plausible to him that a is 
telling the truili rather than !ying: True, P <s Lie, P. 

As aspecific examp!e, consider the scenario in Example 1, and assume 
now that Alice tells Bob (aftel' seeing that the coin was lying Heads up): "I 
saw the face, so now I know: The coin is lying Tails up". Assume that Bob 
trusts Alice completely, so he believes that she is telling the truth. We can 
model this action using the following action model X: 


| 

i Kor a gr | 
hd e 

| 


This model has two actions: the one on the left is the real action that is 
taking place (in which Alice's sentence is a lie: in fact, she doesnit know the 
coin is Tails up), while the one on the right is the other possible action (in 
which Alice is telling the truth: she does know the coin is Tails up). We 
labeled this node with their preconditions, ~K,7 for the lying action and 
KaT for the truth-telling action. In each case, Alice knows what action she 
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is doing, so her information cells (the continuous squares) are singletons; 
while Bob is uncertain, so the dashed square includes both actions. As 
before, we use arrows for plausibility relations, skipping all the loops. As 
assumed above, Alice is not decided to always lie about this; so, a priori, 
she finds her lying in any such given case to be equally plausible as her 
telling the truth: this is refiected by the fact that the continuous arrow is 
bidirectional. In contrast, (Bob's) dashed arrow points only to the node on 
the right: he really believes Alice! 


The product update of two plausibility modeis. We are ready now 
to define the updated (state) plausibility model, representing the way some 
action, from an action plausibility model © = (X, <<, <a, pre)aEA, will act 
on an input-state, from an initially given (state) plausibility model S = 
(S,r-:«, <q, 11.[)aEA' We denote this updated model by S ® &, and we call 
it the update product of the two modeis. Its states are elements (s, ø) of the 
Cartesian product S x X. More specifically, the set of states of S ® & is 


EQF := {(s,0):s E pre(o)s} 


The valuation is given by the original input-state model: for all (s,o) E SQ 
È, we put (s,o) p iff s H} p. As epistemic uncertainty relations, we take 
the product of the two epistemic uncertainty relations|": for (s, o), (si, 0’) E 


S&F, 


(S,0) ~a (3,0) iff o ~a g’, S ~a Si 


Finally, we define the plausibility relation as the anti-lexicogmphic preorder 
relaiion on pairs (s, 0), i.e.: 


(S,0) <a cst. 0") iff either o <a o’ or 6 ~a 0',8 Xo Si. 


In (Baltag and Smets, 2008), we called this type of product operation 
the Action-Prioritu Update, with a term due to .l. van Benthem (personal 
communication). 


Interpretation. To explain this definition, reeall first that we only deal 
with pure "belief changes", not affecting the "facts": this explains our "con- 
servative" valuation. Second, the product construction on the epistemic in- 
distinguishability relation r-:« is the same as in (Baltag and Moss, 2004): if 
two indistinguishable actions are successfully applied to two indistinguish- 
able input-states, then their output-states are indistinguishable. Third, 
the anti-lexicographic preorder gives "priority" to the action plausibility 
relation; this is not an arbitrary choice, but is motivated by our above- 
mentioned interpretation of "actions" as specific types of belief changes. 


13 Observe that this is precisely the uncertainty relation ofthe epistemic update product, 
as defined in (Baltag and Moss, 2004). 
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The action plausibility relation captures what agents realls] believe is going 
on at the moment; while the input-state plausibility relations only capture 
past beliefs. The doxastic action is the one that "changes" the initial doxas- 
tic state, and not vice-versa. If the "believed action" œ requires the agent 
to revise some past beliefs, then so be it: this is the whole point of believing 
a, namely to use it to revise or update one's past beliefs. For example, in 
a successful lying, the action plausibility relation makes the hearer believe 
that the speaker is telling the truth; so she'll accept this message (unless 
contradicted by her knowiedge), and change her past beliefs appropriately: 
this is what makes the lying "successful". Giving priority to action plausibil- 
ity does not in any way mean that the agent's belief in actions is "stronger" 
than her belief in states; it just captures the fact that, at the time of updat- 
ing with a given action, the belief about the action is what is actual, what is 
present, is the curreni belief about what is going on, while the beliefs about 
the input-states are in the past.'* The belief update induced by a given 
action is nothing but an update with the (presently) believed action. 

In other words, the anti-lexicographic product update refiects our Motto 
above: beliefs about changes (as formalized in the action plausibility rela- 
tions) are nothing but ways to ericode changes of belief (i.e., ways to change 
the original plausibility order on states). This simply expresses our partic- 
ular inierpretation of the (strong) plausibility ordering on actions, and is 
thus a matter of convention: we decided to introduce the order on actions 
to eneode corresponding changes of order on states. The product update 
is a consequence of this convention: it just says that a strong plausibility 
order œ <a l3 on actions corresponds indeed to a change of ordering, (from 
whatever Lhe ordering was) between Lhe original input-statee s,f, Lo Lhe 
order (S,a) <a (t, 13) between output-states; while equally plausible actions 
æ ~a lI wilileave the initialordering unchanged: (S,a) <a (tj3) iff s <a t. 
So the product update is just a formalization of our inierpretation of action 
plausibility modeis, and thus it doesn't impose any further limitation to our 
setting. 


Example 3: By computing the update product of the plausibility model S 
in Example 1 with the action model © in Example 2, we obtain the following 
plausibility model: 


C q 
l 

l 

| 

L 


14 Ofcourse, at a later moment, the above-mentioned belief about action (now belonging 
to the past) might be itself revised. But this is another, future update. 
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This correctly describes the effect of an action of "successful lying": Bob's 
plausible ordering is reversed, since he believes Alice, and so now he believes 
that the coin is lying face up. In contrast, Alice's initial plausibility relation 
is unaffected (since the two actions were equally plausible, i.e., doxastically 
equivalent, for her); so, she should keep her a priori belief that the coin is 
Heads up; of course, in this case the last point is not so relevant, since Alice 
knows the state of the coin (as witnessed by the fact that the continuous 
squares consist of single states). 


Example 4: "Hard" public announcements. A iruihful public an- 
nouneement |P of some "hard fact" P is not really about belief revision, 
but about the learning of certified irue information: it establishes common 
knowledge that P was the case. This is the action described by van Ben- 
them (2004) as (public) "belief change under hard facts". As an operation 
on modeis, this is described by van Benthem (2004) as taking any state 
model S and deleting all the non-P states, while keeping the same indis- 
tinguishability and plausibility relaiions between the surounru; states. In our 
setting, the corresponding action model consists of only one node, labeled 
with P. It is easy to see that the above operation on models can be ex- 
actly "simulated" by taking the anti-lexicographic product update with this 
one-node action model. 


Example 5: "Lexicographic upgrade” as a "soft" public announce- 
ment. To allow for "soft" belief revision, an operation fP was introduced 
by van Benthem (2004), essentially adapting to public announcements the 
‘lexicographic’ policy for belief revision described by Rott (1989). This 
operation, called "lexicographic update" consists of changing the cutrent 
plausibility order on any given state model as follows: all P-worlds become 
more plausible than all ~P-worlds, and within the two zones, the old order- 
ing remacns. In our setting, this action corresponds to the following local 
plausibility action model: 


Taking the anti-lexicographic update product with this action will give an 
exact "simulation" of the lexicographic upgrade operation. 


Proposition 3.1. The update product of a state plausibility model and an 
action plausibility model is a state plausibility model. 
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4 Product update in CDM form 


As for the "static" conditional doxastic logic, the axioms of "dynamic" belief 
revision logic can be easily derived if we first work out the CDM version of 
the above update product. We do this from scratch, by first introducing a 
"dynamic" version of the notion of CDM, equivalent to the above concept 
of action plausibility model: 


A conditional doxastic action model (CDAM, [or short} & is just a con- 
ditional doxastic frame (X, {el laca ncs), together with a precondition map 
pre : X — Prop as above. A set of actions H C } can be interpreted as 
pariiel injormaiion about some real (basic) action ø E II, or equivalently, as 
a non-deterministic action (in which one of the actions ø E II happens, but 
we are not told which). The conditional appearance ol captures the way 
action © eppears to agent a, given additional (plausible, but noi tiecessar- 
ily truthful) injormaiion II about this action. This means that, in normal 
circumstances, if aftel' o happens the agent is told that (one of the actions 
in) If has happened, then the agent will believe that in fact (one of the 
basic actions in) ol has happened. As before, any action plausibility model 
induces in a canonieal way a CDAM, and conversely any CD AM can be 
represented as the canonieal CDAM of some action plausibility model. 


Example: Lying, revisited. In the successful lying example, if we con- 
vert the plausibility model into its canonieal CDM, we obtain, e.g., that 
(Lie, P)? = {True, P} for bÆ a and Q Z {Lie, P}. So this lying is in- 
deed genemlly "successful": no matter what other information is given to 
b, if it is consistent with a telling the truth, then b believes that atelIs the 
truth. The only case in which the appearance of this action to bis different 
is when Q C {Lie, P}, in which case (Lie., P)? = Q, and in particular, 
(Lie, py {ties} = {Lie, P}: so the hearer can discover the lying only if given 
information that excludes all other possible actions. 


Independenee of action's appearance from prior beliefs. The above 
description assumes that the agerii's beliefs about the action are independent 
of his beliefs about the state: indeed, oll contains no information about, 
or reference to, the cutrent state's doxastic appearance sa to the agent, 
so it is assumed that this does not inftuence in any way the appearance 
of the action. This assumption embodies a certain interpretation of our 
"appearance" maps: we take an action's appearance to simply denote the 
action itself, as it appears to the agent. In other words: for the agent, the 
appearance is Lhe act.ion, pure and sirnple. When Lhe action ø happens 
(say, in an unconditional context), it realls] appears to the agent as if (the 
apparent, un-conditionalized action) ga := g% happens. Ifthe agent makes 
the additional hypothesis that one of the actions in H happens, then it 
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appears to him that oll is happening. The action's appearance is simply 
taken here as a brute, new fact: the agent really believes this apparent action 
is happening (otherwise this would noi really be the appearance of this 
action). This belief cannot be retnsed at the same time that it is being held: 
any reoision ofthe action's appeamnce can only happen in the future. But for 
the moment, this appearance correctly reftects what the agent thinks to be 
happening. In contrast, his prior beliefs about the state are just that: prior 
beliefs. They may be subject to revision at this very moment, due to cutrent 
action (or, more precisely, due to its appearance): indeed, the (apparent) 
action is the one that induces the revision (or update) of the static belief. In 
a certain sense, the action, as it appears, is the belief update: the apparent 
action simply eneodes the way the agent is compelled to update his prior 
beliefs. Hence, the action's appearance cannot, by definition, be dependent, 
or be inftuenced, by these prior beliefs: the action's appearance is a given, 
it is what it is, and the prior beliefs are the ones that may be changed by 
the apparent action, not vice-versa. 

Taking the action's appearance as a correct description of the action 
as seen by the agent, the above independence (of this appearance from 
prior beliefs) can be understood as a mtionality postulate: agents should be 
prepared to revise their prior beliefs when faced with (what eppears to them 
as) iruihful new information. Rational agents are not fundamentalists: if 
given compelling evidence to the contrary (as encoded in the "apparent 
action" ), they will not refuse it due to prior beliefs, but will change these 
prior beliefs to fit the new evidence. And it does not matter in the least 
that, at some later point, this "compelling evidence" might turn out to have 
been a belief (an "apparent acr.ion"), noL a realiLy: when this will happen, 
rational agents might change their minds again. But for the moment, they 
have to accept the cutrent action as it eppears to them, and adjust their 
previous beliefs appropriately. 


An action's contextual appearance. In the context of belief revision, 
there is a subtie point to be made here: the above independence only refers 
to the agent's prior beliefs, but not to the agent's knowiedge. No action's 
eppearanee can be assumed to be independent of prior knowiedge: it might 
happen that the cutrent state s is such that agent a knows that the believed 
action gl cannot happen at this state. This is perfectly possible, even in 
states in which a does happen, and even if the information II is correct (i.e., 
a E ID. In such a state, the agent cannot accept the default appearance 


elt, Prior knowledge may thus inftuence the action's appearance. 


Example 2, revisited: "Successjul" lying cannot always be successful! 
Indeed, if the original input-state s is such that an outsider b alreeds] knows 
that P is false, then lying cannot succeed. In this context, the appearance 
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of the action Lie, P to bis noi its default appearance True, P: b cannot 
believe that a is telling the truth. Instead, the contextual appeamnce of this 
action at s is itself: (Lie., P)b = Lie, P. The hearer knows the speaker is 
lying. 

So oll should only be thought of as the action's default appeamnce (con- 
ditional on 11) to the agent: in the absence of any other additional infor- 
mation (except for 11), or uiheneoer the agent's prior knowledge allows this, 
the agent a will believe that ol has happened. 

So how will this action appeal' in a context in which the default appear- 
ance is known to be impossibie? We can answer this question by defin- 
ing a contextual appeamnce gêl of action a to agent a at state s, given 
1l. We can do this by strengthening our conditionalization: at a given 
state s E S, an agent has already some information about the next ac- 
tion, namely that it cannot be inconsistent with his knowledge s(a) of the 
state. In other words, agent a knows that the action must belong to the 
set Xs(a) := {p E X : s(a) N pre(p)s 4 O} = {p E È : s Kg Ka—pre(p)}. 
Putting this information together with the new information 11, we obtain 
the contextual appearance by conditionalizing the aqent's belief about the 
action with ¥Xs{a) N 11: 

gl = oon _ pal El:s(a)pre(o) #2} 
This contextual appearance is the one that fully captures the agent's actual 
belief about the action a in state s, whenever he is given information 11. 


An action's effect: Deterministic change of state. As announced, we 
take the basic actions a E X} to represent deterministic changes of states. 
In the following, we will always represent the output-state a(s) of applying 
basic action a to state s E S by an ordered pair a(s) := (s,0). So, for 
a given CDM S of possible input-states and a given CDAM of possible 
actions, the set of all possible output-states will be a subset of the Cartesian 
product S x X. Thus, we could represent post-conditions, i.e., conditions 
restricting the possible output-states of some (unspecified) action acting 
on some (unspecified) input-state as subsets P C S x X} of the Cartessari 
product. Given a basic action a E and a post-condition P C Sx¥, we may 
denote the set of possible input-states of action a ensuring post-condition 
P by: 
o '(P)= {s ES: o(s) EP} = {s ES: (s,o) EP} 


Post-conditional contextual appearance. Sometimes, the additional 
information the agent may be given (or the hypothesis that he may enter- 
tain) refers, not directly to the range 11 C © of possible actions currently 
happening, but to some post-condition P; i.e., the agent might be told that 
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the cutrent action will result in a state o(S)= (S,c) satisfying some post- 
condition P C S x X. He should be able to conditionalize his belief about 
the cutrent action with this information, in a given context. For this, we de- 
a ,P . . . 
fine the contextual eppearanee o? of action 0 at state s, in the hypothesis 
(that the action wil! ertsure postcondition) P, by putting: 


oh? = o {PEh:3(a)no'(P)AB} 


Example: lying, again. Let again Lie, P be the action of successfullying 
by agent a, and suppose that P denotes afactual ("ontic") statement (which 
happens to be false, and thus will remain false aftel' lying). Even if in the 
original state, the hearer b did not know that P was false (so that lying 
was successful, and its appearance to b was the default one True, P), he 
may be given later that information, as a post-condition ~P. Then, the 
hearer discovers the lying: the post-conditional contextual appearance of 
lying (given ~P) is... lying! 


Belief revision induced by an action and a postcondition. We want 
to calculate now the reoisiori of an agerii's beliefs (about an input-state 
s) induced by an action o when given some post-condition P C Sx È. 
We denote this by go, This captures the appeamnce of the input-state 
s to agent a, afier action © and ajier being given the informaiion that P 
holds at the output-state. As explained already, the agent revises his prior 
beliefs not in accordance with the actual action, but in accordance to how 
this action appears to him. As we have seen, the appearance of action 0 
at state s when given post-condition P is ger So the new information 
obtained post-factum about the original input-state s is that this state was 
capable of supporting (one of the actions in) git and moreover that it 
yielded an output-state satisfying post-condition P. In other words, the 
agents learns that the original state was in (08) ~1(P). So he has to revise 
(conditionalize) his prior beliefs about s with this information, obtaining: 


oP @ she SE UP) 
Sa’. aT 


Product update, in CDM form. We now give a CDM equivalent of the 
above notion of product update: the product update of a conditional doxastic 
model S with a conditional doxastic action model © is a new conditional 
doxastic model S ® X, whose states are elements o(S) := (S,o) of a subset 
SQX of the Cartesian product Sx». Note that we prefer here the functional 
notation o(s), instead of (s, o). As before, preconditions select the surviving 
states: 


SÈ := {o(s):s E pre(o)s} 
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For any hypothesis P C S & X} about the output-state, the conditional 
appearance (conditioned by P) of an output-state o{s) to an agent a is 
given by 


o(s)e = 04" (87) nP 


In words: Agent a's updated belief (about the output-state of a basic ac- 
tion o applied to an input-state s, when given condition P) UOH can be 
obtained by applying the action that is believed to happen (i.e., the appear- 
ance gi? of g to a at s, given post-condition P) to the agent's reoised belief 
about the input-state sor (belief revised with the information provided by 
the apparent action ga? 


Finally (as for plausibility modeis), the valuation on output-states comes 
from the original states: 


lpllsos := {o(s) ES £ : s EllpllS} 


), then reetricting to the given post-condition P. 


Proposition 4.1. The two "product update" operations defined above 
agree: the canonical CDM associated to the (anti-lexicographic) product 
update of two plausibility models is the product update of their canonieal 
CDM's. 


5 The dynamic logic 


As in (Baltag and Moss, 2004), we consider a doxastic sumature, i.e., a fi- 
nite (fixed) plausibility frame (or, equivalently, a finite conditional doxastic 
frame) X, together with an ordered list without repetitiotie (01, ... , On) of 
some of the elements of X. Each signature gives rise to a dynamic-doxastic 
logic CDL(‘X), as in (Baltag and Moss, 2004): one defines by double recur- 
sion a set of sentences p and a set of program terms T; the basic programs 
are of the form it = of = oY... Pn, where o E X and y; are sentences 
in our logic; program terms are generated from basic programs using non- 
determirusiic sum (choice) nUn’ and sequential composition Tin’, Sentences 
are built using the operators of CDL, and in addition a dynamic modal- 
ity (w)y, taking program terms and sentences into other sentences. As in 
(Baltag and Moss, 2004), the conditional doxastic maps on the signature 
X induce in a natural way conditional doxastic maps on basic programs 
in CDL(X): we put (og)! := {o'g :o' E oll}. The given listing can 
be used to assign syntactic preconditions for basic programs, by putting: 
pre(oig) := yi, and pre(og) := T (the trivially true sentence) if ø is not 
in the listing. Thus, the basic programs of the form og form a (finite) syn- 
tactic CDAM" Xv. Every given interpretation 1111 : CDL(X) — PTOp of 


15 A syntactic COAM is just a conditional doxastic frame endowed with a syntactic 
precondition map, associating sentences to basic action. For justification and examples, 
in the context of epistemic action models, see (Baltag and Moss, 2004). 
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sentences as doxastic propositions will convert this syntactic model into a 
"reai" (semantic) CDAM, called X|]. 

To give the semantics, we define by induction two inierpretation maps, 
one taking any sentence y to a doxastic proposition ||y|| E PTOP, the sec- 
ond taking any program term & to a (possibly non-deterministic) doxastic 
"program", i.e., a set of basic actions in some CDAM. The definition is 
completely similar to the one in (Baltag and Moss, 2004), so we skip the 
details here. Suffice to say that the semantics of basic dynamic modalities 
is given by the inverse map: 


IA vlls = (elpis) ‘Hl goxysy 


Notation. To state our proof system, we encode the notion of post-condi- 
tional contextual appeamnce of an action in our syntax. For sentences ý, % 
and basic program & = oY, we put: 


w= V Cara N -kapya N Kantoo ) 
TCE BETI Bet 


This notation can be justified by observing that it semantically matches the 
modality corresponding to post-conditional contextual appearance: 


I(oP)2)¢lls = [s ES: SE GER E 


Theorem 5.1. A complete proof system for the logic CDL(X) is obtained 
by adding to the above axioms and rules of CDL the following Reduction 
Axioms: 


(mUm)p = (nje V (re 
(mnp = (nye 
(a)p = pre(a) Ap 
(ajy = prela) A r{ajp 
(aey y) = (ap V (ay 
(a)B2p > prea) A BEE? [a2] (8 > p) 


where pis any atomie sentence, zr, 7’ are programs and & is a basic program 


in DCS). 


The soundness of the last reduction axiom is obvious, once we see that its 
holding at a state s follows immediately from the definition of the product 
update in CDM form 


ele = 94!" (sg?) OP, 


by taking o := a, P := ||| and using the semantics of dynamic modalities. 
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Special cases. If we put the last reduction axiom in its dual (universal 
modality) form, we obtain 


[o|B°y © pre(a) > Bio? lat] (0 > p). 


As special cases of the Action-Conditional-Belief Law, we can derive the 
redmetion laws from (van Benthem, 2004) for (conditional) belief aftel' the 
events l and fW: 


ly]B?y = y => B Idle, 
[MYB e = (KYO a BYAT fele) v GEM A BM tylo) 
where K?) := K,( > 0), K¥0 := nK 0, BYO := =B} ~. 
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Abstract 


I first summarize Baltag and Smets' contribution to this volume, and 
praise their work. Then I compare the anti-lexicographic plausibility 
update that they propose to a proposal by Aucher, as an illustration 
of the difference between a qualitative and a quantitive formulation of 
updates. I quote Spohn's original work that is at the root of research 
in plausibility updates and of the notion of anti-lexicographic update. 
Some technical notes on different partial orders used in belief revision 
serve as a prelude to an observed relation between the qualitative and 
quantitative representation of structures. Finally I address Baltag 
and Smets' analysis of the action of lying. 


In this commentary on 'The Logic of Conditional Doxastic Actions' I am in 
the delightful position of having the last word in an argument with Alexan- 
dru Baltag. This position is very hard to obtain. But because in this volume 
my commentary follows the chapter by Alexandru Baltag and Sonja Smets, 
any further elaborations and involutions will be out of reach to the readers 
of the volume. lam going to use this rare advantage to the limit. 

Having said that, I sent a preliminary version of these comments to 
Alexandru and Sonja for comments, and immediately received in response 
an email of about the same length as this submission. I am very grateful 
for Alexandru's last words. Weil, nearly last words. I made some further 
changes. Now, the work is done. 


1 The logic of conditional doxastic actions 


In AGM belief revision a distinction is made between belief expansion and 
(proper) belief revision. Given a set of consistent beliefs, in belief ex pan- 
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sion new information can typically be added as such, without conflict with 
existing beliefs. But in belief revision, the incoming information is typi- 
cally inconsistent with the prior beliefs. A long line of work in dynamic 
epistemic logic, prominently including the well-known framework of action 
modeis, also by Baltag but with different collaborators, can be seen as a 
generalization of belief expansion. Unlike AGM belief revision in its origi- 
nal formulation, this dynamic epistemic logic also models belief expansion 
for more than one agent, and what is known as higher-order belief change: 
given explicit operators for 'the agent believes that', self-reference to one's 
own belief or to the beliefs of others can also be formalized. A problem 
in that line of research remained that the typical belief reoision, i.e., how 
to process inconsistent new beliefs, cannot be modelled. Belief in factual 
information, for example, cannot be given up when confronted with new 
belief that is considered as acceptable evidence to the contrary. And this 
is not just impossible within the setting of knowiedge, where one does not 
expeet proper revision to be possible, because knowledge is truthful. It is 
also impossible for weaker epistemic notions. 

In this contribution to the volume, Alexandru Baltag and Sonja Smets 
introduce adynamie epistemic framework in which belief revision in the 
proper sense is, aftel’ all, possible. Given a structure (an epistemic plausi- 
bility frame) wherein one does not merely have epistemic indistinguishability 
between states but also plausibility relations between states, one can define 
both knowledge and conditional belief operators. Unconditional belief is 
defined as belief that is conditional to the trivial state of information. (The 
trivial state of information is the epistemic equivalence class occupied by 
Lhe agent, which is described by Lhe Iorrnula T.) In this seu.ing belief re- 
vision is possible where the agent (unconditionally) believed some factual 
information p but aftel' having been presented with convincing evidence to 
the contrary, changes his mind, and then believes the exact opposite. Just 
as for the relation between classic AGM expansion and dynamic epistemic 
logic, we now have again that this approach also models muiti-agent and 
higher-order belief revision. 

The authors go much further, beyond that. In a multi-agent setting 
there are more complex forms of belief revision than revision with a publicly 
announced formula y. That is merely an example of a doxastic action. 
More complex doxastic actions, where the action appears differently to each 
agent, are also conceivable. They present a very good example, namely the 
action where agent a is lying that p. To a credulous agent b, this action 
will appeal’ as a truthful announcement that y. But not to a of course, 
who knows that she is Iying. The general form of doxastic actions is like 
an epistemic plausibility model and is called an action plausibility model; 
the difference is that instead of a valuation of atoms in each state of an 
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epistemic plausibility model we now have a precondiiion for the execution 
of each action (i.e., element of the domain) of an action plausibility model. 
The execution of such a doxastic action in a epistemic plausibility state 
is a restricted modal product, where I am tempted to say 'as usual', to 
avoid the obligation of having to explain this in detail. The only unusual 
aspect of this procedure is the very well-chosen mechanism to compute 
new plausibilities from given plausibities, called anti-lexicogmphic preorder 
relaiion. This says that plausibility among actions takes precedence over 
plausibility among states. It is the natural generalization of the implicit 
AGM principle that the revision formula takes precedence over the already 
believed formulas. Anti-lexicographic preorder prescribes that: a new state 
of affairs is more plausible than another new state of affairs, if it results 
from an action that is strictly more plausible than the action from which 
the other state results, or if the states result from equally plausible actions 
but the former state already was more plausible than the latter state before 
action execution. 

So far, this overview also describes the authors' other publication (Bal- 
tag and Smets, 2006). A main focus of their underlying contribution is the 
interpretation of these results in terms of conditional reasoning and con- 
ditional doxastic action models. The conditional eppearanee maps of these 
conditional doxastic action models take the place of the plausibility rela- 
tions among the actions in an action plausibility model. They motivate 
and justify in great detail various notions for conditional belief, and their 
interdependencies. A fabulous finale is a complete axiomatization with a 
reduction axiom that relates conditional belief aftel' an action to conditional 
belief befere that action. The technicalit.ies of this logic wiLh dynarnic op- 
erators for conditional action execution may be hard to follow unless the 
reader is intimately familiar with the BMS action model framework, as these 
technical details are only somewhat summarily presented. In that case, just 
focus on this reduction axiom, the action-conditional-beliej law, and the 
well-chosen examples given of its application. I can assure you, it's all true. 


2 Quality is better than quantity 


The authors claim that their approach is 'in its spirit closer to qualitative 
logics than to approaches of a more quantitative ftavour.' This is a very 
well-considered phrasing. Let us see why they are right. 

One such approach of a more quantitative ftavour is the proposal by 
Guillaume Aucher (2003). In terms of action plausibility models his pro- 
posal is a version of van Benthem's soft update referred to in Example 5, but 
with a different recipe to compute new plausibilities. Aucher also employs 
structures with epistemic equivalence classes and plausibility relations, but 
the plausibility relations are derived from a finite total order of degrees of 
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plausibility. If s is more plausible than si, its degree of plausibility is lower. 
The lowest degree of plausibility is 0. To each degree of plausibility corre- 
sponds a degree of belief, expressing that the believed proposition is at least 
as plausible as that. Unconditional belief is belief of degree 0. 

Given some model with domain S, Aucher's ‘belief revision with a for- 
mula y’ now amounts to the following. Whenever y is true, subtract the 
minimum degree of the cp-states from the current degree. Otherwise, sub- 
tract the minimum degree of the —y-states from the cutrent degree and add 
one. This ensures that at least one cp-state will get degree 0, and thus factual 
information ~ will be unconditionally believed aftel’ revision, as required. 

For an example, consider one agent a only and a domain consisting of 
four states 0, 1,2,3 comprising a single equivalence class (all four states are 
considered possible by the agent) and such that 0 <a 1 <a 2 <a 3. In this 
initial epistemic plausibility structure, the degree of each state is its name. 

First, suppose that factual information p is true in state 3 only (the 
valuation of p is {3}). According to the recipe above, the result is the 
order 3 <a 0 <a 1 <a 2. How come? Write dg(S) for the old degree of 
state s and dg'{s) for its new degree, aftel' revision. Then dg = dg(3) - 
Min{dg(s) Is Hp} = 3-3 = 0. Whereas dg'{1) = dg(1) -Min{dg(s) Is 4 
p}+1= 1- 0+1= 2. Etcetera. So far so good. 

Now for some other examples, demonstrating issues with such quantita- 
tively formulated proposals for belief revision. Suppose that, instead, p is 
true in states 1 and 2. We now obtain that 1<,2~,0 <3. Asa result of 
this revision, states 2 and 0 have become equally plausible. As a side effect 
of the revision, such ‘loss of plausibility information’ may be considered less 
desirable. 

Finally, suppose that p was already believed: suppose that p is true in 
states 0 and 1. We then get 0 <a 1 =a 2 < 3. This is also strange: instead 
of reinforcing belief in p, the ~p-states have become more plausible instead! 

This example demonstrates some issues with a quantified formulation of 
belief revision. Of course Aucher is aware of all these issues. See Aucher's 
PhD thesis (2008) for a quite novel way to perform higher-order and multi- 
agent belief revision, based on plausibility relations among sets of formulas 
describing the structure in which the revision is executed. 


3 Belief revision known as maximal-Spohn 


When I first heard from Baltag and Smets' work on plausibility reasoning my 
first response was: "But this has all been done already! It's maximal-Spohn 
belief revision!" Aftel' some heavy internal combustion, I told Alexandru, 
who has his own response cycle, and this is all a long time ago. At the time 
I thought to remember specific phrasing in Spohn's well-known ‘Ordinal 
Conditional Functions' (1988). But I never got down to be precise about 
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this souree of their work. Now lam. This section of my comments can be 
seen as yet another footnote to the extensive references and motivation of 
the authors' contribution. Intheir Example 5 they explain that for revision 
with single formulas all the following amount to more or less the same: anti- 
lexicographic update, lexicographic update, soft public update. To this list 
we can add yet another term: what Hans Rott (in his presentations and in 
Rott, 2006) and I call 'maximal-Spohn belief revision’. 

Spohn introduces the 'simple conditional functions' (SCF) and the equiv- 
alent notion of 'well-ordered partitions’ (WOP) to represent the extent of 
disbelief in propositions. In terms of Baltag and Smets, a WOP defines 
a totally ordered plausibility relation on the domain. Spohn then observes 
that such WOPs (plausibility relations) need to be updated when confronted 
with incoming new information in the form of a proposition A. In our terms 
A is the denotation of some revision formula y. He then proceeds to discuss 
some specific plausibility updates. His presentation is based on ordinals 
a, 8,7,'" that label sets Ey, Eg, Ey, ... of equally plausible states (all the 
Ea-states are more plausible than all the E{3-states, etc.). For a simplifying 
example, consider a partition of a domain W into a well-ordered partition 
Eo, El, ...,E5. The set Eo are the most believed states, etc. Assume that 
a proposition A has non-empty intersection with E4 and Es. Thus, the 
most plausible A-states are found in E,. If we now also read 'state x is less 
plausible than state y' for 'world x is more disbelieved than world y’ we 
are ready for an original quote from (Spohn, 1988), explaining two different 
ways to adjust El,...,F6 relative to A. A clear sign of a great writer is 
that one can take his work out of context but that it remains immediately 
inLelligible. 


A first proposal might be this: It seems plausibie to assume that, 
after information A is accepted, all the possible worlds in A are less 
disbelieved than the worlds in A (where A is the relative comple- 
ment W\ A of A). Further, it seems reasonable to assume that, 
by getting information only about A, the ordering of disbelief of the 
worlds within A remains unchanged, and likewise for the worlds in A. 


(Spohn, 1988, pp. 112-113) 


(... ) the assumption that, after getting informed about A, all worlds 
in A are more disbelieved than all worlds in Aseems too strong. Cer- 
tainly, the first member, i.e. the net content of the new WuP, must 
be a subset of A; thus at least some worlds in A must get less disbe- 
lieved than the worlds in A. But it is utterly questionable whether 
even the most disbelieved world in A should get less disbelieved than 


even the least disbelieved world in A; this could be effected at best 
by the most certain information. 


This last consideration suggests a second proposal. Perhaps one 
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should put only the least disbelieved and not all worlds in A at the 
top of the new WOP (...). (Spohn, 1988, pp. 113-114) 


The first proposal has become known as maximal-Spohn. The second pro- 
posal has become known as minimal-Spohn. Applied to the example par- 
tition the results are as follows; on purpose I use a fotmatting that is veTY 
similar to the displayed formulas on pages 113 and 114 in (Spohn, 1988). 
Further down in the sequence means less plausible. 


E4nA, Es nA, Eo,El,E2,E3,E4nA,EsnA maximal-Spohn 
E4N A, Eo, El, E2, E3, E4N A, Es minimal-Spohn 


In maximal-Spohn, as in antilexicographic update, the A-states now come 
first, respecting the already existing plausibility distinctions among A-states, 
so that we start with E4n A, Es nA. The order among the non-A-states 
also remains the same (whether intersecting with A or not), thus we end 
with Eo, El, E2, E3, E4nA, Es OA. In minimal-Spohn, the states in Es are 
not affected by proposition A; only the equivalence class containing most 
plausible A-states is split in two, and only those most plausible A-states, 
namely E4n A, are shifted to the front of the line. These are now the most 
plausible states in the domain, such that A is now (in terms of Baltag and 
Smets again) unconditionally believed. 

Aucher's plausibility update (Aucher, 2003), that we discussed in the 
previous section, implements a particular kind of 'minimal-Spohn' that also 
employs Spohn's ordinal conditional functions. We do not wish to discuss 
those here-things are quantitative enough as it is, already. Aucher's is 
not as minimal as it can be, e.g., I demonstrated the side-effect of merging 
plausibilities. It would be interesting to see a truly qualitative form of 
plausibility update that amounts to minimality in the Spohn-sense, or at 
least to something less maximal than anti-lexicographic update but equally 
intuitively convincing; but I do not know of one. 


4 Well-preorders 

Inepistemic plausibility frames (5, r-:« <,)ac, the epistemic indistinguisha- 
bility relations are equivalence relations and the plausibility relations are 
required to be well-preorders, i.e., refiexive and transitive relations where 
every non-empty subset has minimal elements. The non-empty-subset re- 
quirement ensures that something non-trivial is always conditionally be- 
lieved. I will clarify some technical points concerning these primitives, to 
illustrate their richness for modelling purposes. 


On the meaning of minimal. A weil-order is a total order where every 
non-empty subset has a least element, so by analogy a well-preorder should 
indeed be, as Baltag and Smets propose, a pre-order where every non-empty 
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subset has minimal elements. So far so good. Right? Wrong! Because we 
haven't seen the definition of minimal yet. I thought I did not need one. 
For me, an element is minimal in an ordered set ij nothing is smaller: 


Min(<,, S) := {s If e S and t <a simplies t ~, s}. 


Also, their first occurrence of 'minimal' before that definition was in a fa- 
miliar setting, 'something like a weil-order', so that | did not bother about 
the precise meaning. And accidentally the definition of minimal was not 
just aftel' that first textual occurrence but even on the next page. So | had 
skipped that. 

This was unwise. For Baltag and Smets, an element is minimal in an 
ordered set ij eveTything is biggeT: 


Min™(<,, 5) = {s | t € S implies s <a t}. 


This is a powerful device, particularly as for partial orders some elements 
may not be related. The constraint that all non-empty sets have minima in 
their sense applies to two-element sets and thus enforces that such relations 
are connected orders (as explained in Footnote 4 of Baltag and Smets' text). 
So every well-preorder is a connected order. On connected orders the two 
definitions of minimal (Min'" and Min) coincide. We can further observe 
that the quotient relation <,/<q is a total order, and that it is also a well- 
order. Given a non-empty subset S' of <,/<a, there is a non-empty subset 
S” of <, such that S" \ œa = S', The ~,-equivalence class of the minimal 
elements of S” is the least element of S’, The well-preorders of the authors 
are sometimes known as templated orders (Meyer et al., 2000). All this 
corresponds to Grove systems of spheres, as the authors rightly state. 


Partial orders in belief revision. Partial orders that are not connected 
or not well-ordered according to the authors’ definition do occur in belief 
revision settings. From now on I will only use 'minimal' in the standard 
sense. 

Given one agent a, consider the frame consisting of five states {O, 1, 2, 3, 
4}, all epistemically indistinguishable, and such that the relation <, is 
the transitive and refiexive closure of O <a 1 < 4 and 0 <a 2 <a 3 <a 
4. It is a partial order, and every non-empty subset has minima. The 
reader can easily check this, for example, Min(<g, {0, 1,2,3,4}) = {O}, 
Min(<a, {1, 2,3) = {1,2} = Min(<g, {1,2}), and so on. If neither s <q t 
nor t <g s, states s and ¢ are called incompamble. States 1 and 2 are 
incomparable, as are 1 and 3. 

Consider the symmetrie closure Sy(<,) of a plausibility relation <, that 
is a partial order and where every non-empty subset has minima. Given a 
state s, we call a state t ~a s plausible iff (s, t) E Sy(<,). Conditionalization 
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to implausible but epistemically possible states is clearly problematic. So 
as long as all states in an equivalence class are plausible regardless of the 
actual state in that class, we are out of trouble. This requirement states 
that the epistemic indistinguishability relation ~a must be a refinement of 
Sy(<,), or, differently said, that ~a NSy(<,)= <<. 

Incomparable states in a partial order can be 'compared in a way’ aftel' 
all. Define s =, t iff for all u E S, u. <a s iff u. <a t. Let's say that the agent 
is indijJeTent between s and ¢ in that case. Clearly, equally plausible states 
are indifferent: ~a C =a. But the agent is also indifferent between the 
incomparable states 1 and 2 in the above example. The quotient relation 
<,/=a is a total order. In belief contraction this identification of incom- 
parabie objects in a preorder typically occurs between sets of formulas, not 
between semantic objects. See work on epistemic entrenchment involving 
templated orders, e.g., (Meyer et al., 2000). 


Qualitative to quantitative. As already quoted by me above, the au- 
thors consider their approach 'in its spirit closer to qualitative logics than to 
approaches of a more quantitative ftavour.' Well-chosen wording, because- 
as the authors know-in its ruiture their approach is fairly quantitative aftel' 
all. Let us see why. 

From a preorder where all non-empty subsets have minimal elements we 
can create degrees of plausibility as follows. Given that all sets have minimal 
elements, we give the <,-minimal states of the entire domain S degree of 
plausibility 0. This set is non-empty. Now the entire domain minus the 
set of states with degree 0 also has a non-empty set of minimal elements. 
Again, this set exists. These are the states of degree 1. And so on. Write 
Degree’(<,) for the set of states of degree i. We now have: 


Degree? (<4) Min(<a, S) 
Degree*t"(<,) Min(<,, S\ Uj=0.k Degree*(<,)) 


Note the relation with the total order <,/=, introduced above. 

Ofcourse, I entirely agree that a qualitative preseniatioti of an epistemic 
plausibility framework is to be preferred over a quantitative representation. 
And-this is once again Alexandru Baltag providing an essential comment 
to the preliminary version of this commentary-although this comparison 
can be made on the structural level, the language of conditional doxastic 
logic is apparently not expressive enough to define degrees of belief, that 
use the above order. This matter is explained in their related publication 
(Baltag and Smets, 2006). But with that result one can wonder if a weaker 
structural framework, more qualitative in nature, would already have suf- 
ficed to obtain the same logical result. 

It seems to me that the quest for the nature and the spirit of qualitative 
belief revision has not yet been ended. Other frameworks for belief revision, 
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such as the referenced work by Fenrong Liu, her PhD thesis (2008), and my 
own work (van Ditmarseh, 2005) (where the non-empty minima requirement 
is only for the entire domain, thus al!owing the real number interval [0,1 D, 
sometimes employ other partial orders and basic assumptions and mayalso 
contribute to this quest. 


5 This is a lie 


The authors’ analysis of "lying about y” involves an action plausibility 
model consisting of two actions Lie.,(y) and True,(y) with preconditions 
aK,ay and Kay respectively. These actions can be distinguished by the 
lying agent, the speaker a, but are indistinguishable for the target, the 
listener b. Further, b considers it more plausible that a speaks the truth, 
than not: Truea(cp) <», Liea(cp). So 'agent a lies about y’ means that a 
armounces that is true, thus suggesting that she knows that ¢ is true, 
although a does in fact not know that. For convenience | am presenting 
this action as a dynamic operator that is part of the language (which can 
be justified as the authors do in Section 5). 

In the authors' subsequent analysis it is explained how the contextual 
appearance of an action mayalso determines its meaning, both the context 
of states wherein the action may be executed and the context of states 
resulting from the action's execution. Again, 'lying about y’ makes for a 
fine example. If the listener balready knows that y is false, the act of lying 
does not appeal’ to bas the truth that y, but as a lie that y. 


| have two observations to this analysis. 


Lying and bluffing. 1 think that the precondition of a ‘lying that y’ is 
not ignomnce of the truth, but knowledge to the contrary: the precondition 
of the action Liea(cp) should not be ~Kyy but K,-y. If the precondition is 
AK ay instead, | cal! this bluffing, not lying. As | am not a native speaker 
of English, and neither are the authors, this seems to be as good a moment 
as any to consult a dictionary (Merriam-Webster). To blufJ is "to cause to 
believe what is untrue." Whereas to lie is "to make a statement one knows 
to be untrue." It is further informative to read that the etymology for 
‘bluff’ gives "probably from the Dutch bluffen, for 'to boast’, 'to play a kind 
of card game." It is of course typical Anglo-Saxon prejudice that al! bad 
things concerning short-changing, scrooging, boasting, diseases, and unfair 
play (‘Dutch book') are called Duteh. But let's not pursue that matter 
further. Given the action Truea(cp), that expresses the for b more plausible 
alternat.ive, I t-hink that its precondit.ion Kay properly expresses Lhe part 
‘to cause to believe what is untrue’. On the other hand, given that the 
action Lie.,(y) that is considered less plausible by b, the precondition K,7y 
seems to express accurately 'to make a statement one knows to be untrue,’ 
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and this condition is stronger than the precondition ~Kgy suggested by 
Baltag and Smets. 

When I proposed this commentary to the authors, Alexandru Baltag 
came with an interesting response: the precondition “Ky of action Liea(y) 
also involves 'to make a statement one knows to be untrue’, namely the 
statement 'I know ip’, In fact, a knows that she does not know y. This is 
true. But a bit further-fetched, if I may. For me, the prototypical example 
of a lie remains the situation where, way back in time, my mother asks 
me if I washed my hands before dinner and I say: "Yes." Whereas when 
my grandfather held up his arm, with a closed fist obscuring a rubber (for 
Americans: eraser) and asked me: "What have I got in my hand?" and I 
then respond "A marbie!" he never accused me of being a liar. Or did he? 
I'd like to investigate these matters further. I am unaware of much work 
on lying in dynamic epistemics. For a setting involving only belief and not 
knowiedge, and public but not truthful announcements, see (van Ditmarsch 
et al., 2008). 


Is contextual appearance relevant? I question the need for contextual 
appearances of actions. I make my point by resorting to lying, again. The 
authors say that the precondition of a lie is ~K,y but that, if the listener b 
already knows that y is false, the act of lying no Ionger appears to bas the 
truth that ọ, but as a lie that y. I would be more inclined to strengthen 
the precondition for lying about ọ from =Kyy to 7KgpgAn7K p-7y. In which 
case there is no need for this contextual precondition. 

Combining this with the previous I therefore think that the precondition 
of Lie,(y) should be Kary A aKy7y rather than “Ky. And this is only 
the beginning of a more and more fine-grained analysis of lying, not the 
end. For example, it is reasonable to expect that the speaker is aware of the 
listener's ignorance about y. That makes yet another precondition, namely 
K,7kK,-7y. A distinction between knowledge and belief mayalso be impor- 
tant to model lying. The typical convention is to assume common belief 
that the speaker is knowledgeable about y but the listener not, although in 
fact the speaker knows (or at least believes) the opposite of y; so we get 


Kary A CBap(Bo( Kay V Kany) A 7Koy A 7K p7y) 


where CB is the common belief operator. We cannot replace common be- 
lief by common knowledge in this expression. Then it would be inconsis- 
tent. (We can also replace all other K-operators in this expression by B- 
operators.) There are also truly multi-agent scenarios involving lying, where 
only the addressee is unaware of the truth about y but other listeners in 
the audience may have a different communicative stance. 

If this is only the beginning and not the end, why should there be an 
end at all? It is in fact unclear (as Alexandru Baltag also mentioned in 
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response to reading a version of these comments) if by incorporating more 
and more 'context' we finally have taken all possible contexts into account. 
Maybe there will always turn up yet another scenario that we might also 
want to incorporate in the precondition of lying. On the other hand-me 
again trying to have to last word-it seems that by employing infinitary 
operators in preconditions such as common knowledge and common belief, 
as above, we can already pretty weil take any kind of envisaged variation 
into account. So my cutrent bet is that the preconditions of contextual 
appearances (not the postconditional aspect) can be eliminated altogether. 

I am detracting myself, and the reader. So let me stop here. Does this 
show that the authors’ analysis of lying is ftawed? Not at all! In fact it is 
very weil chosen, as it is a very rich speech act with many hidden aspects 
that are crying aloud for analysis, and the authors' framework of doxastic 
actions is the obvious and very suitable formalization for such an analysis. 
Also, different arguments than the above can be put forward, in support 
of =Kay as precondition of Lie,(y) instead of my preferred Kamy. Let 
me therefore conclude by complimenting the authors again on their rich 
contribution, and hope for more from this productive duo. 
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Abstract 


We study a branching-time temporallogic of belief revision where the 
interaction of belief and information is modeled explicitly. The logic 
is based on three modal operators: a belief operator, an informa- 
tion operator and a next-time operator. We consider three logics of 
increasing strength. The first captures the most basic notion of min- 
imal belief revision. The second characterizes the qualitative content 
of Bayes' rule. The third is the logic proposed in (Bonanno, 2iiii7a), 
where some aspects of its relationship with the AGNI theory of be- 
lief revision were investigated. We further explore the relationship to 
AG NI with the help of semantic structures that have been used in the 
rational choice literature. Further strengthenings of the logic are also 
investigated. 


1 Introduction 


Since the foundational work of Alchourrén, Gärdenfors and Makinson 
(1985), the theory of belief revision has been a very active area of research. 
Recently several authors have been attempting to re-cast belief revision 
within a modal framework. Pioneering work in this new area was done 
by Segerberg (1995, 1999) in the context of dynarnic doxastic logic, Board 
(2002) in the context of multi-agent doxastic logic and van Benthem (2004) 
in the context of dynarnic epistemic logic. Much progress has been made 
both in dynarnic episternic logic (see, for example, Baltag and Smets, 2006; 
van Ditmarseh, 2005; van Ditmarsch and Labuschagne, 2007 and the recent 
survey in van Ditmarsch et al., 2007) as weil as in dynamic doxastic logic 
(see Leitgeb and Segerberg, 2007). Another very active area of research has 
been iterated belief revision (see, for example, Boutilier, 1996; Darwiche and 
Pearl, 1997; Nayak et al., 2003; Rott, 2006). 

This paper joins the recent attempts to establish a qualitative view of be- 
lief revision in a modal framework, by continuing the study of belief revision 
within a temporal framework that was first proposed in (Bonanno, 2007a). 
Since belief revision deals with the interaction of belief and information over 
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time, branching-time temporal logic seems a natural setting for a theory of 
belief change. On the semantic side we consider branching-time frames with 
the addition of a belief relation and an information relation for every in- 
stant 7. We thus extend to a temporal setting the standard (Kripke, 1963) 
semantics used in the theory of statie belief pioneered by Hintikka (1962). 
On the syntactic side we consider a propositionallanguage with a next-time 
operator, a belief operator and an information operator. Three logies of 
increasing strength are studied. The first is a logic that expresses the most 
basic notion of minimal belief revision. The second captures the qualitative 
content of Bayes' rule, thus generalizing the two-date result of (Bonanno, 
2005) to a branching-time framework. The third logic is the logic proposed 
in (Bonanno, 2007a), where some aspects of the relationship between that 
logic and the AGM theory of belief revision were investigated. In this paper 
we provide frame characterization results for al! three logies and we further 
investigate the relationship between the strongest of the three logies and 
the notion of AGM belief revision functions. We do so with the help of 
semantic structures that have been used in the rational choice literature. 
We cal! these structures one-stage revision frames and show that there is a 
correspondence between the set of one-stage revision frames and the set of 
AGM belief revision functions. Further strengthening of the logic are also 
investigated. 

While the structures that we consider accommodate iterated belief revi- 
sion in a natural way, we do not attempt to axiomatize iterated revision in 
this paper. First steps in this direction have been taken in (Zvesper, 2007). 

We provide frame characterization results and do not address the issue 
of cornpleteness of our logics. Cornpleteness of Lhe basic logic wiLh respect 
to a more general class of temporal belief revision frames (where the set of 
states is allowed to change over time) is proved in (Bonanno, 2008); that 
result has been extended in (Zvesper, 2007) to the set of frames considered 
in this paper. 


2 Temporal belief revision frames 


We consider the semantic frames introduced in (Bonanno, 2007a), which 
are branching-time structures with the addition of a belief relation and an 
information relation for every instant t. 

A next-time bmnching frame is a pair (T, >) where T is a non-empty, 
countable set of instants and > is a binary relation on T satisfying the 
following properties: Vt,,f2,t3 E T, 


(1) backward uniqueness if t:> t3 and t:> t3 then t: = t: 


(2) acyclicity if (tl, ... tn) is a sequence with ti => ti+1 
for every i = 1,...,n -1, then t, Æ tı. 
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The interpretation of t-: > t-: is that t-: is an immediate successar of t-: 
or tı is the immediate predecessor of t : every instant has at most a unique 
immediate predecessor but can have several immediate successors. 


Definition 2.1. A temporal belief reoisiori frame is a quadruple (T, >, Q, 
{Bt Ti}cer) where (T, —) is a next-time branching frame, 2 is a non-empty 
set of states (or possible worlds) and, for every t E T, B, and T, are binary 
relations on Q. 


The interpretation of wBtw’ is that at state wand time ¢ the individual 
considers state io’ possible (an alternative expression is "co' is a doxastic 
alternative to w at time t"), while the interpretation of w/tw' is that at 
state wand time ź, according to the information received, it is possible that 
the true state is io’, We shal! use the fol!owing notation: 


Btw) = {wi EQ: wBtw/! and, similarly, It(w) = {w EQ: witw/!- 


Figure | il!ustrates a temporal belief revision frame. For simplicity, in 
al! the figures we assume that the information relations L; are equivalence 
relations (whose equivalence classes are denoted by rectangles) and the belief 
relations $; are serial, transitive and euclidean' (we represent this fact by 
enclosing states in ovals and, within an equivalence class for /;, we have 
that-for every two states wand wi---wi E Bt(w) if and only if w belongs 
to an oval)." For example, in Figure 1 we have that Z;,(y) = {a,8,y} and 
Bi, (7) = {a, P}. 

Temporal belief revision frames can be used to describe either a situation 
where the objective facts describing the world do not change - so that only 
the beliefs of the agent change over time - or a situation where both the 
facts and the doxastic state of the agent change. In the literature the first 
situation is cal!ed belief revision, while the latter is cal!ed belief update (see 
Katsuno and Mendelzon, 1991). We shal! focus on belief revision. 

On the syntactic side we consider a propositional language with five 
modal operators: the next-time operator O and its inverse O71, the belief 
operator B, the information operator land the "al! state" operator A. The 
intended interpretation is as follows: 


O¢: "at every next instant it wil! be the case that ¢” 
Q-'¢@: "at every previous instant it was the case that œ” 
Bo: "the agent believes that ¢” 

lọ: "the agent is informed that o” 

Ag : "it is true at every state that @”. 


1 Bt is serlal if, Vw E Q, Bt(w) Æ O; it is transitive if w' E Bt(w) implies that Bt(w') C 
Bt(w); it is euclidean if w' E Bt(w) implies that Bt(w) C Bt(w'). 

2 Note, however, that our results do not require T, to be an equivalence relation, nor do 
they require Bt to be serial, transitive and euclidean. 
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The "all state" operator A is needed in order to eapture the non-normality 
ofthe information operator / (see below). For a thorough discussion ofthe 
"all state" operator see (Goranko and Passy, 1992). 

Note that, while the other operators apply to arbitrary formulas, we 
restriet the informaiion operator to apply to Boolean [ormulas only, that 
is, to formulas that do not eontain modal operators. Boolean formulas 
are defined reeursively as follows: (1) every atomie proposition is a Boolean 
formula, and (2) if éand % are Boolean formulas then so are ~g and (V4). 
The set of Boolean formulas is denoted by $Ë. Boolean formulas represent 
faets and, therefore, we restriet information to be about faets. 3 

Given a temporal belief revision frame (T, >, Q, {Bt,Z.}:e7) one obtains 
a model based on it by adding a function V : S — 2° (where S is the set of 
atomie propositions and 2° denotes the set of subsets of Q) that assoeiates 
with every atomie proposition p the set of states at whieh p is true. Note 
that defining a valuation this way is what frames the problem as one of belief 
revision, sinee the truth value of an atomie proposition p depends only on 
Given a model, a state w, an instant tand 
a formula ¢, we write (w, t) | ¢ to denote that ¢ is true at state wand 
time 4 Let ||¢|| denote the truth set of ¢, that is, ||d|] = {(w EQxT: 
(w, t) | $} and let [dlr C Q denote the set of states at whieh ¢ is true at 
time 4, that is, [|]: = {w E€ Q : (w,t) | >}. Truth of an arbitrary formula 
at a pair (w, t) is defined reeursively as follows: 


the state and not on the time.’ 


3 Zvesper (2007) has recently proposed a version of our logic where the restrietion to 
Boolean forrnulas is dropped. 
4 Belief update would require a valuation to be defined as a function V :S — DXT, 


Belief Revision in aTemporal Framework 49 


ifp ES, (w,t) = p if and only if wE V(p); 
w, t) Ea if and only if (w,t) É ¢; 
(w,t) =V if and only if either (w,t) E ¢ or (w, t) EK w (or both); 
(w,t) FO¢ if and only if (w,ti) | œ for every ti such that t > ti; 
(wt) EQ -'¢ if and only if (w,til) = @ for every til such that til — t; 
w, t) = Bo if and only if Btw) C [¢],, that is, 
if (WI, t) =| ¢ for all io’ E Bt(w); 
(w,t) | Id if and only if Jt(w) = [@],, that is, if (1) (W, t) EK ¢ 
for all w E It(w), and (2) if w, t) = @ then io’ E It(w); 
(w, t) Ad if and only if øl; = Q, that is, 

if (WI, t) | @ for all io E Q. 


Note that, while the truth condition for the operator B is the standard 
one, the truth condition for the operator / is non-standard: instead of 
simply requiring that Jt(w) C [¢], we require equality: It(w) = [¢],. Thus 
our information operator is formally similar to the "all and only" operator 
introduced in (Humberstone, 1987) and the "only knowing" operator studied 
in (Levesque, 1990), although the interpretation is different. It is also similar 
to the "assumption" operator used by Brandenburger and Keisier (2006). 


Remark 2.2. The truth value of a Boolean formula does not change over 
time: it is only a function of the state. That is, fix an arbitrary model and 
suppose that (w,t) = ġ where @ E OF; then, for every ti E T, (w,ti) = ¢ġ 
(for a proof see Bonanno, 2007a, p. 148). 


A formula ¢ is valid in a model if ||ġ|| = Q x T, that is, if œ is true at 
every state-instant pair (W,t). A formula ¢ is valid in a frame if it is valid 
in every model based on it. 


3 The basic logie 


The formal language is built in the usual way (see Blackburn et al., 2001) 
from a countable set of atomie propositions, the connectives -, and V (from 
which the connectives A, — and + are defined as usual) and the modal 
operators O, O-', B, 1 and A, with the restrietion that Jọ is a well- 
formed formula if and only if ¢ is a Boolean formula. Let O¢ #_0 ad, 
and ~to  -,O-1'-7¢. Thus the interpretation of Od is "at some next 
instant it will be the case that œ” while the interpretation of O-1¢ is " 
some immediately preceding instant it was the case that œ”. 

We denote by ILa the basic logic dcfincd by the following axioms and 
rules of inference. 


at 
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Axioms: 
1. All propositional tautologies. 
2. Axiom K for O,O-', Band A> for D E {O,O-1,B,A}: 


(Og A0(¢ > 4)) > Oy (K) 


3. Temporal axioms relating O and O-'. 


¢ 7-00 '¢ 
7-0 'O¢ 
4. Backward Uniqueness axiom: 
0 '¢+0''¢ (BU) 
5. 85 axioms for A: 
Ag > 


AA¢ — AAAd 
6. Inclusion axiom for B (note the absence of an analogous axiom for 1): 
Agd > Bo (IndB) 


7. Axioms to capture the non-standard semantics for I: for ġ, 7 E ®? 
(recall that $Ë denotes the set of Boolean formulas), 


HoA Ib) > Aloo p) 
Alg = 4) = (I¢ = Ty) 
Rules of Inference: 
1. Modus Ponens: 
$, ġġ >y (MP) 
y) 
2. Necessitation for A, O and O-': 
For every D E t, 0, 0-'), 5: (Nec) 


5 Axiom K for / is superftuous, since it can be derived from axioms lı and 12 below (see 
Bonanno, 2005, p. 204). 
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Note that from MP, Inclj, and Necessitation for A one can derive ne- 
cessitation for B. On the other hand, necessitation for J is noi a rule of 
inference of this logic (indeed it is not validity preserving). 


Remark 3.1. By MP, axiom K and Necessitation, the following is a derived 
rule of inference for the operators 0 „O-1 Band A: of tly for DE 
{O,O-1,B,A}. We call this rule RK. On the other hand, rule RK is not 


avalid rule of inference for the operator J. 


4 The weakest logic of belief revision 


Our purpose is to model how the beliefs of an individual change over time in 
response to factual information. Thus the axioms we introduce are resiricted 
to Boolean [ormulas, which are formulas that do not contain any modal 
operators. 

We shall consider axioms of increasing strength that capture the notion 
of minimal change of beliefs. 

The first axiom says that if é and w are facts (Boolean formulas) and- 
currently-the agent believes that ¢ and also believes that % and his belief 
that ¢ is non-trivial (in the sense that he considers œ possible) then-at 
every next instant-if he is informed that ¢ it will still be the case that he 
believes that.é. That is, if at a next instant he is informed of some fact that 
he currently believes non trivially, then he cannot drop any of his cutrent 
factual beliefs ('W' stands for 'Weak' and 'ND' for 'No Drop'):6 if ¢ and w 
are Boolean, 


(Bọ A =B=¢ A BY) > Olg > BY). (WND) 


The second axiom says that if ¢ and w are facts (Boolean formulas) 
and-currently-the agent believes that œ and does not believe that p, 
then-at every next instant-if he is informed that ¢ it will still be the case 
that he does not believe that %. That is, at any next instant at which he 
is informed of some fact that he currently believes he cannot add a factual 
belief that he does not currently have ('W' stands for 'Weak' and 'NA' 
stands for 'No Add'):" ¢ and w are Boolean, 


(B6A -BY) > OUg > =B4). (WNA) 


6 lt is shown in the appendix that the following axiom (which says that if the individual 
is informed of some fact that he believed non-trivially at a previous instant then he 
must continue to believe every fact that he believed at that time) is equivalent to 
WND: if and w are Boolean, 


0-!(Bẹ N By AAB-=¢) A Ib > Bu. 


This, in turn, is propositionally equivalent to O-! (Bẹ A By \ ABA) —> (Id —> BY). 
7 It is shown in the appendix that the following is an equivalent formulation of WNA: 
if ġ and W are Boolean, 


OTIB A ABW) A Ib > ~By. 
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Thus, by WND, no belief can be dropped and, by WNA, no belief can 
be added, at any next instant at which the individual is informed of a fact 
that he currently believes. 

An axiom is characterized by (or characterizes‘ a property of frames if 
it is valid in a frame if and only if the frame satisfies that property. 

Ali the propositions are proved in the appendix. 


Proposition 4.1. 


(1) The axiom WND is characterized by the foliowing property: Yw E Q, 
Vty,t2 E T, 


iff] => 12,B,,(w) £0 and B,,(w) < 14.(w) then B,o(w) < N., (w). 
(Pwno) 


(2) The axiom WN A is characterized by the foliowing property: Vw E Q, 
Vti, t-- E T, 


if ti > tz and By, (w) C Te (w) then Ba (w) C By, (w). (Pwna) 


Let ILw (where 'W' stands for 'Weak') be the logic obtained by adding WND 
and WNA to ILa. We denote this by writing ILw = ILa+ WNA+ WND. The 
foliowing is a coroliary of Proposition 4.1. 


Corollary 4.2. The logic ILw is characterized by the class of temporal 
belief revision frames that satisfy the foliowing property: Vw E Q, Vty,t:E 
T, 


ifti — 12, B,,(w) £0 and B,,(w) C 1,ə(w) then B, (w)= B(w). 


The frame of Figure 1 violates the property of Coroliary 4.2, since t-: > t3, 
Bi(a) = {a} C 149(a)= {0,6} and Bula) = {8} £ Bela). 

The logic ILw captures a weak notion of minimal change of beliefs in 
that it requires the agent not to change his beliefs if he is informed of some 
fact that he already believes. This requirement is stated explicitly in the 
foliowing axiom ('WNC' stands for 'Weak No Change’): if ¢ and w are 
Boolean formulas, 


(1¢éA 07! (B¢ A A=B->¢)) = (By = O71 BY). (WNC) 


WNC says that if the agent is informed of something that he believed non- 
trivialiy in the immediately preceding past, then he now believes a fact if 
and only if he believed it then. 


Proposition 4.3. WNC is a theorem of ILw. 


We now turn to a strengthening of ILw. 
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5 The logic of the Qualitative Bayes Rule 


The logic ILw imposes no restrictions on belief revision whenever the indi- 
vidual is informed of some fact that he did not previously believe. We now 
consider a stronger logic than ILy . The foliowing axiom strengthens WND 
by requiring the individual not to drop any of his cutrent factual beliefs 
at any next instant at which he is informed of some fact that he currently 
conciders possible (without necessarily believing it: the condition B¢ in the 
antecedent of WND is dropped): if ¢ and % are Boolean, 


(AB-$ A Bb) > O(I¢ > BY). (ND) 


The cortesponding strengthening of WNA requires that if the individual 
considers it possible that (/ A ~y) then at any next instant at which he is 
informed that ¢ he does not believe that 7:3 if œ and % are Boolean, 


AB>A(d A mw) > Olg = ~By). (NA) 


One of the axioms of the AGM theory of belief revision (see Gärdenfors, 
1988) is that information is believed. Such axiom is often referred to as 
"Success" or "Acceptance". The foliowing axiom is a weaker form of it: 
information is believed when it is not surprising. If the agent considers a 
fact @ possible, then he wili believe ¢ at any next instant at which he is 
informed that ¢. We cali this axiom Qualified Acceptance (QA): if ¢ is 
Boolean, 


AB7A¢ — O(l¢ > Bẹ). (QA) 


Proposition 5.1. 


(1) The axiom ND is characterized by the foliowing property: Vw E Q, 
Vti,t: ET, 


if tı > t and B, (w) OTa (w) Æ Ø then B,,(w) C By, (w). (Pyp) 


(2) The axiom NA is characterized by the foliowing property: Vw E Q, 
VWti,t2 ET, 


if ti — ty then Ba (w) N Te (w) C Bi, (w). 


8 Axiom NA can alternatively be written as O([¢ A Bw) — B( — w), which says 
that if there is a next instant at which the individual is informed that @ and believes 
that %, then he must now believe that whenever @ is the case then w is the case. 
Another, propositionally equivalent, formulation of NA is the following: a~B(¢@ —> 
p) += O(lé — ~By), which says that if the individual does not believe that whenever 
@ is the case then w is the case, then-at any next instant-if he is informed that ¢ 
then he cannot believe that %. 
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(3) The axiom QA is characterized by the foliowing property: Vw E Q, 
Ve, t2E T, 


if tı — to and Ba (w) OTe (w) A Ø then Be (w) C Tn lw). (Poa) 


We cali the foliowing property of temporal belief revision frames "Qual- 
itative Bayes Rule" (QBR): Vt,,t-: ET, Ww E Q, 


if t4 — tz and B, (w) NT, (w) # Ø then Ba (w) = By, (w) OTe (w). (QBR) 


The expression "Qualitative Bayes Rule" is motivated by the foliowing ob- 
servation (see Bonanno, 2005). In a probabilistic setting, let Pw,t, be the 
probability measure over a set of states Q representing the individual's be- 
liefs at state wand time f,; let F C Q be an event representing the in- 
formation received by the individual at a later date t2 and let Put, be 
the posterior probability measure representing the revised beliefs at state 
wand date 12. Bayes' rule requires that, if Pw,t, (F) > 0, then, for every 


event E < Q, Py. (E) = B COR, 


(where supp(P) denotes the support of the probability measure P): 


Bayes' rule thus implies the foliowing 


if supp(Pw,t,) Nn F # 0, then supp(P.,.4,) = supp(Pw,t,) n F. 


If we set Bt, (w) = supp(Pw,t,), F = Te (w) (with +: > 12) and Bt2(w) = 
sUPP(Pw,t2) then we get the Qualitative Bayes Rule as stated above. Thus 
in a probabilistic setting the proposition "at date ¢ the individual believes 
é” would be interpreted as "the individual assigns probability 1 to the event 


ll <Q”. 


The foliowing is a coroliary of Proposition 5.1. 


Corollary 5.2. The conjunction ofaxioms ND, NA and QA characterizes 
the Qualitative Bayes Rule. 


The frame of Figure 1 violates QBR, since tz > t3, By2(0) = {8, y} and 
T (ô) = {y,6,e}, so that Ba (8) O Te (ô) = {y} A 0; however, By, (ô) = 
{y, 0} Æ Br2(0) N Ta (ô). On the other hand, the frame of Figure 2 does 
satisfy QBR. 


Definition 5.3. Let ILogr = La + ND +NA +QA. 


Remark 5.4. The logic Lape contains (is a strengthening of) ILw. In 
fact, WND is a theorem of the logic ILa + ND, since (BAA =B7¢d A BY) > 
(-B-7¢ /\ Bw) is a tautology, and WN A is a theorem of the logic ILa + NA 
(see the appendix). 
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FIGURE 2. 


6 The logic of AGM 


We now strengthen logic ILogr by adding four more axioms. 
The first axiom is the Acceptance axiom, which is a strengthening of 
Qualified Acceptance: if œ is Boolean, 


I¢— Bo. (A) 


The second axiom says that if there is a next instant where the individual 
is informed that ¢l\.w and believes that X, then at every next instant it must 
be the case that if the individual is informed that ¢ then he must believe 
that (¢ nN. Y) — X (we call this axiom K7 because it corresponds to AGM 
postulate (®7): see the next section): if d, 7 and X are Boolean formulas, 


OU (GN. Y) \. BX) = Olg = B ((¢ n. 4) > X)). (K7) 


The third axiom says that if there is a next instant where the individual 
is informed that ¢, considers #l\.y) possible and believes that y — X, then at 
every next instant it must be the case that if the individual is informed that 
nN. % then he believes that X (we call this axiom K8 because it corresponds 
to AGM postulate (@8): see the next section): if ¢, ọ and X are Boolean 
formulas, 


Olen aBA(¢ n y) BY > X)) = OU (G1. y) > BX). (K8) 


The Iourt.h axiorn says Lhat if Lhe individual receives consistent infor- 
mation then his beliefs are consistent, in the sense that he does not si- 
multaneously believe a formula and its negation ('WC' stands for 'Weak 
Consistency'): if œ is a Boolean formula, 


56 G. Bonanno 


(I$ A An$) > (BY > ~B). (WC) 
Proposition 6.1. 


(1) The axiom (A) is characterized by the foliowing property: Vw E Q, 
WET, 
B(w) C (w). 


(2) The axiom (K7) is characterized by the foliowing property: Yw E Q, 
Vt1, t2, t3 ET, 


ifti => t2, t] = t3 and Te (w) C Ten (w) 
then Z; (w) N Bt2(w) < Br3(w). (Px7) 


(3) The axiom (K8) is characterized by the foliowing property: Yw E Q, 
Vti, t2 t3 E T, 


ifti > 12,t] > 13, Ti,(w) C Tu (w) and Te (w) nB R(w) #0 
then Bz3(w) < Tea (w) NB12(w). (Prs) 


(4) The axiom (WC) is characterized by the foliowing property: Yw E Q, 
WET 
if (w) Æ Ø then B(w) £ Ø. (Pwc) 


Let LAGW = La +A +ND +NA +K7 +K8 + We. Since QA can be 
derived from A, logic ILAG] contains (is a strengthening of) logic Lager. 


Definition 6.2. An ILAGIlvI-fmme is a temporal belief revision frame that 
satisfies the foliowing properties: 


(1) the Qualitative Bayes Rule, 

(2) Ww EQ, YtE T, Btw) C Ti(w), 

(3) Yw E Q, YŁE T, if Z,(w) #0 then Bw) £ O, 
(4) Ww EQ, Yti tata ET, 


ifti => 12, tl 13, Ta, (w) C Tu (w) and Te (w) nBt2(w)# 0 
then B,3(w) = Ta, (w) N B(w). 


An ILAGIvl-model is a model based on an ILAGIvI-frame. 
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The frame of Figure 2 is not an AGM frame, although it satisfies QBR. 
In fact, we have that ty > t2, ty > t3, Teny) = {7,6}, Z.(y) = {8,7,6} 
and Ba (7) = {6,1}, so that 14a(7) E 142(7) and tyo(7) Ba (7) = {7} #0 
but Bely) = {7,0} Æ tas (W) NBi2(wW) = {7}. 


Corollary 6.3. It follows from Proposition 6.1 that logic ILAG is charac- 
terized by the class of ILAGIvI-frames. 


Some aspects of the relationship between logic ILAGI and the AGM 
theory of belief revision were investigated in (Bonanno, 2iiii7a). In the 
next section we explore this relationship in more detail, with the help of 
structures borrowed from the rational choice literature. 


7 Relationship to the AGM theory 


We begin by recalling the theory of belief revision due to Alchourron, 
Gärdenfors and Makinson (1985), known as the AGM theory (see also 
Gärdenfors, 1988). In their approach beliefs are modeled as sets of formulas 
in a given syntactic language and belief revision is construed as an operation 
that associates with every deductively closed set of formulas K (thought of 
as the initial beliefs) and formula ¢ (thought of as new information) a new 
set of formulas K9 representing the revised beliefs. 


7.1 AGM belief revision functions 


Let S be a countable set of atomie propositions and Lo the propositional 
language built on S. Thus the set Po of formulas of Lo is defined recursively 
as follows: if p ESthen p E ®ọ and if ¢,7%) E Po then ~ E ®p and 
PVWE Do. 

Given a subset K C 4p, its PL-deductive closure [K]PH (where 'PL' 
stands for Propositional Logic) is defined as follows: % E [APs if and only 
if there exist ¢,,... ¢, E K such that (¢4 A... A n) > Y is a tautology 
(that is, a theorem of Propositional Logic). A set K C ®o is consistent if 
[K]P4 4 ®o (equivalently, if there is no formula ¢ such that both ¢ and 7d 
belong to [K]P}). A set K C ®o is deductively closed if K = ar, A belief 
set is a set K C Po which is deductively closed. The set of belief sets will 
be denoted by K and the set of consistent belief sets by K°°”. 

Let K € K be a consistent belief set representing the agent's initial 
beliefs. A belief reoisiori function fOT K is a function 


K® : po + 2% 


that associates with every formula ¢ E Po (thought of as new information) 
a set K®() C Po (thought of as the new belief). It is common in the 
literature to use the notation Ke instead of K®(¢), but we prefer the latter. 
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A belief revision function is called an AGM reoision function if it satisfies 
the following properties, known as the AGM postulates: V¢,~ E Bo, 


K®(¢) eK (@1 
pE KË(¢) (®2 
K? ($) < [K U {0} (@3 
if ad ¢ K, then [K U{d}]?" < K®(¢) (@4 
if œ is a contradiction then K®() = ®o (@5a 
if $ is not a contradiction then K®(¢) 4 Bo (@5b 
if ġ w is a tautology then K®(¢) = K®(w) (@6 
Ke(or $) < [KOAU 0] (97 
if ~p ¢ K(g), then [KE(4) U {y} < KPA). (88 


®1) requires the revised belief set to be deductively closed. 
®2) requires that the information be believed. 


®3) says that beliefs should be revised minimally, in the sense that no 
new formula should be added unless it can be deduced from the information 
received and the initial beliefs. 


®4) says that if the information received is compatible with the initial 
beliefs, then any formula that can be deduced from the information and the 
initial beliefs should be part of the revised beliefs. 

@®5ab) require the revised beliefs to be consistent, unless the information 
œ is contradictory (that is, ~ is a tautology). 

®6) requires that if d is propositionally equivalent to % then the result 
of revising by ¢ be identical to the result of revising by w. 

@®7) and (@8) are a generalization of (@3) and (@4) that 


"applies to itemted changes of belief. The idea is that if K®(¢) is a 
revision of K [prompted by ¢] and K®(6) is to be changed by adding 
further sentences, such a change should be made by using expansions 
of K®() whenever possible. More generally, the minimal change of 
K to include both ¢ and w (that is, K®(¢Aw)) ought to be the same 
as the expansion of K®(d) by Y, so long as Y does not contradiet the 
beliefs in K®(¢)” (Gärdenfors, 1988, p. 55).9 


We now turn to a semantic counterpart to the AGM belief revision 
functions, which is in the spirit of Grove's (1988) system of spheres. The 
structures we will consider are known in rational choice theory as choice 
functions (see, for example, Rott. 2001; Suzumura, 1983). 


9 The expansion of K® ($) by w is [KE ($) U {¥}] PE, 
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7.2 Choice structures and one-stage revision frames 


Definition 7.1. A choice struciure is a quadruple (Q, E,O,R) where 


e Q is a non-empty set of states; subsets of Q are called eoenis. 
e £ C 2È? is a collection of events (oe denotes the set of subsets of Q). 


e R : E — 2° is a function that associates with every event E E € an 
event RE C Q (we use the notation RE rather than R(E)). 


- OEE is a distinguished element of E with OF o. 


In rational choice theory a set E E € is interpreted as a set of available 
alternatives and RE is interpreted as the subset of E which consists of those 
alternatives that could be rationally chosen. In our case, we interpret the 
elements of € as possible items of information that the agent might receive 
and the interpretation of RE is that, if informed that event E has occurred, 
the agent considers as possible all and only the states in RE. For the 
distinguished element ©, we interpret Ro as the originol or initial beliefs 
of the agent.I" 

Note that we do not impose the requirement that QE E. 


Definition 7.2. A one-stage remsioti frame is a choice structure 
(Q, €,O, R) that satisfies the following properties: VE, F € €, 


RE CE, (BRI) 
if E~0 then RE £0, (BR2) 
if E C F and RF nE <0 then RE = RFNE, 11 (BR3) 
if Ron E #0 then RE = Ron E. (BR4) 


In the rational choice literature, (BR1) and (BR2) are taken to be part of 
the definition of a choice function, while (BR3) is known as Arrow's axiom 
(see Suzumura, 1983, p. 25). Property (BR4), which corresponds to our 
Qualitative Bayes Rule, has not been investigated in that literature. 


The following is an example of a belief revision frame: Q =- {c, 13, y, 5}, 
E = {{o, 3}, {7,5}, {a 81), O={0, 1377, Rias) = 13} Ros} = {1}, 
Riasan = j3) 
10 In the rational choice literature there is no counterpart to the distinguished set O. 


11 It is proved in the appendix that, in the presence of (BRI), (BR.3) is equivalent to: 
VE, FEE, 


if REN E#0 and En FEE then REnF = RFN E. (BR.3') 
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A one-stage reoisiori model is a quintuple (Q,€,0,R, V) where (0,€, 0, 
R) is a one-stage revision frame and V :S — 2° is a function (called a 
valuation) that assoeiates with every atomie proposition P the set of states 
at whieh P is true. Truth of an arbitrary formula in a model is defined 
reeursively as follows (w |= ¢ means that formula ¢ is true at state w): (1) 
for PE S, w }p if and only if w E V(p), (2) w = -¢ if and only if w A ¢ 
and (3) w = @V¥w if and only if either w E ¢ or w E w (or both). The 
truth set of a formula ¢ is denoted by I¢||. Thus lo] = fw EQ: w H o}. 


Given a one-stage revision model, we say that 
(1) the agent initially believes that ¢ if and only if Ro C I¢||, 


(2) the agent believes that } upon learning that w if and only if ||| E E 
and Rijy\ <|l¢ll- 


Definition 7.3. A one-stage revision model is comprehensioe if for every 
formula ¢, Idl| E €. It is ricli if, for every finite set of atomie propositions 
P =({PIl,...,Pn,ql,---,qmj, there is a state wp € Q sueh that urp |= Pi for 
every | 1,...,n and wp = „gj for every j = 1,...,m. 


Thus in a eomprehensive one-stage revision model every formula is a 
possible item of information. For example, a model based on a one-stage 
revision frame where € = 2° is eomprehensive. In a rieh model every 
formula eonsisting of a conjunction of atomie proposition or the negation of 
atomie propositions is true at some state. 


7.3 Correspondenee 


We now show that the set of AGM belief revision functions eorresponds to 
the set of eomprehensive and rieh one-stage revision modeis, in the sense 
that 


(1) given a eomprehensive and rieh one-stage revision model, we ean as- 
soeiate with it a consistent belief set K and a corresponding AGM 


belief revision function Ke, and 


(2) given a consistent belief set K and an AGM belief revision function 
K® there exists a eomprehensive and rieh one-stage revision model 
whose associated belief set and AGM belief revision function eoineide 
with K and K®, respectively. 


Proposition 7.4. Let (Q,€,O,R, V) be a comprehensive one-stage revi- 
sion model. Define K = {7 E $o : Ro C ||v||}. Then K is a consistent 
belief set. For every ¢ E ®o define KE(¢) = {4 E ®o : Rygy C ||Pll}. Then 
the function K® : Po — 2®° so defined satisfies AGM postulates (@1)- 
(@5a) and (@6)—(@8). If the model is rieh then also (@5b) is satisfied. 
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Proposition 7.5. Let K € K be a consistent belief set and K® : By > 
2%0 be an AGM belief revision function (that is, K® satisfies the AGM 
postulates (@1)-(@8)). Then there exists a comprehensive and rich one- 
stage revision model (0,€,0,R, V) such that K = {4 E ®o : Ro C ||¥||} 
and, for every 6 E Bo, K®(¢) = {v E Bo : Rygy C || YI}. 


7.4 Back to ILAG frames 
Given an ILAGM frame (7,>>,2, {Bt Zi}ter) (see Definition 6.2) we can 
associate with every state-instant pair (wo, fo) a one-stage revision frame 


(see Definition 7.2) (Q°,€°,O°, Ra) as foliows. Let to = {t ET: to — t), 
then 


e Y=, 


E°- {ECQ:E = It(wo) for some t E to}, 


O° = Te (wo), 
e Roo = Bto(wo), 


e for every E E E, if E = It(wo) (for some t e to) then Ri, = Bt(wo). 


By Property (2) of ILAGIvI-frames the frame (2°, E°, O°, Ra) so defined 
satisfies property BR1 of the definition of one-stage revision frame, while 
Property (3) ensures that BR2 is satisfied, Property (4) ensures that BR3 
is satisfied and Property (1) ensures that BR4 is satisfied. 

Consider now the subset of the set of IL AG] frames consisting of those 
frames satisfying the foliowing properties: 


Vt ET, Wwe O,VE E2°\{o},3t’ ET: t ti and Ty(w)= E. (P CMP) 
Vt E T, Ww E 0,7; (w) £ O. (seriality of T4) 


Let Leomp ("comp" stands for "comprehensive") be ILAG + CMP + leon 
where CMP and Icon are the foliowing axioms: for every Boolean ¢ 


nA} > OL¢, (CMP) 
al(¢ A ~o). (Icon) 
Axiom CMP says that, for every Boolean formula ¢, if there is a state 


where ¢ is true, then there is a next instant where the agent is informed 
that œ, while axiom Icon rules out contradictory or inconsistent information. 
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Proposition 7.6. The logic IL.om,) is characterized by the class of ILagw- 
frames that satisfy Pcivip and seriality of 7, .12 


We can view logic lLcomp as an axiomatization of the AGM belief revision 
functions. In fact, if we take any model based on a Loom, frame and any 
state-instant pair, the one-stage revision frame associated with it is such 
that E =- 2°\{@}. Thus the corresponding one-stage revision model is 
comprehensive (see Definition 7.3) and therefore, by Proposition 7.4, the 
associated AGM belief revision function K® : By — 2° satisfies AGM 
postulates (®1)-(@5a) and (@6)—(@8). Conversely, by Proposition 7.5, for 
every consistent belief set K and AGM belief revision function K® : gy > 
2®0 there is a model based on an ILeomp frame whose associated AGM belief 
revision function coincides with K® .13 


Models of IL.omp, however, are "very large" in that, for every state- 
instant pair and for every Boolean formula ¢ whose truth set is non-empty, 
there is a next instant where the agent is informed that ¢. This requirement 
corresponds to assuming a complete belief revision policy for the agent, 
whereby the agent contemplates his potential reaction to every conceivable 
(and consistent) item of information. In a typical [Lagi frame, on the 
other hand, the items of information that the individual might receive at 
the next instant might be few, so that the agent's belief revision policy is 
limited to a few (perhaps the most likely) pieces of information. How does 
this limited belief revision policy associated with Lagi frames relate to the 
AGM postulates for belief revision? The answer is given in the foliowing 
proposition, which was proved in (Bonanno, 2iiti7a) (we have reworded it 
to fit the set-up of this section). We can no longer reeover an entire AGM 
belief revision function from a model based on an arbitrary Lagi, frame. 
However, we can recover, for every pair of Boolean formulas œ and w, the 
values K®(¢) and K®(dA%) of an AGM belief revision function whenever 
there is a next instant at which the agent is informed that œ and there is 
another next instant where he is informed that (¢/\ %). 


Proposition 7.7. 
(A) Let K C ÐP be a consistent and deductively closed set and let K® 
Po — 2% be an AGM belief revision function. Fix arbitrary ¢,7 E ÐP. 


12 Note that, given the non-standard validation rule for [@ the equivalence ofaxiom D 
(I — ~I) and seriality of Z, breaks down. It is still true that if Z, is serial then the 
axiom I[¢@ — aI-7¢ is valid, but the converse is not true (see Bonanno, 2005, p. 226, 
Footnote 25). 

13 All we need to do in this respect is to eliminate the empty set from € in the proof of 
Proposition 7.5, that is, discard the possibility that ¢ is a contradiction. 
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Then there is an ILAGIvI-model, +i, t2, t3 ET and a EQ such that: 


t:> t-: (A.1) 

K = {X E P : (a, tı) K BX} (A.2) 

(a, t2) H Id (A.3) 

K®(¢) = {X E ÐP : (a,12) | BX} (AA) 

if @ is consistent then (8,t) | ¢ for some 8 EQ and t ET (A.5) 
ti =t} (A.6) 

(a, ta) H ICAA Y) (A.7) 

KE(ġ ^y) = {X E ®® : (atz) K BX} (A.8) 


if (ġ 1\ 4%) is consistent 
then (Î' ti) | (@A~w) for some yE Q and ti E T. 


(B) Fix an ILAGIlvI-model such that (1) for some t;, t-i, t3 E T, œ E Q and 
p, Y E pP, tı = tz tı => t3, (a, t2) = I@ and (a, t3) = I(dA wv), (2) 
if ¢ is not a contradiction then (6,t) = œ, for some 8 E Q and t E T 
and (3) if (6 A w) is not a contradiction then (y,ti) | (6 A %), for some 
yE Qand ti ET. Define K = {X E ÐP : (a,t;) K BX}. Then there ex- 
ists an AGM belief revision function K® : Po — 2% such that K®(¢) = 
{X E DP : (a,12) | BX} and K®(¢A%) = {X E © : (a, 13) K BX}. Fur- 
thermore, for every ọ@, 4% E pP, there exists an ILAGIvI-model such that, for 
some & E Q and t2,t3 ET, (1) (@,t2) F Iġ and (a,t3) H I($ A^ %), (2) if 
¢ is not a contradiction then (8,t) = ¢, for some 8 E Q and t ET and (3) 
if (ġ Aw) is not a contradiction then (y,ti) = (6A), for some y E Q and 
ti ET. 


8 Conclusion 


We proposed a temporal logic where information and beliefs are modeled 
by means of two modal operators land B, respectively. A third modal 
operator, the next-time operator O, enables one to express the dynamic 
interaction of information and beliefs over time. The proposed logic can be 
viewed as a temporal generalization of the theory of static belief pioneered 
by Hintikka (1962). 

The combined syntactic-semantic approach of modal logic allows one to 
state properties of beliefs in a clear and transparent way by means ofaxioms 
and to show the correspondence between axioms and semantic properties. 
Natural extensions of our ILAGy logic would impose, besides consistency of 
information (axiom Icon) 4, the standard KD45 axioms for belief (axiom 4: 


14 As pointed out by Friedman and Halperu (1999), it is not clear how one could be 
informed of a contradiction. 
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Bé — BBéand axiom 5: ~B — BaABg¢, while the D axiom: Bọ > ~B7¢d 
would follow from axioms Icon and WC). Furthermore, one might want to 
investigate axioms that capture the notion of memory or recall, for instanee 
Bọ > OBO- IBS and aB¢ — OBO !=B¢ (the agent always remembers 
what he believed and what he did not believe in the immediately preceding 
past). Further strengthenings might add the requirement that information 
be correct (Iġ — ¢) or the weaker requirement that the agent trusts the 
information souree (B O (1¢ — ¢)). Another natural direction to explore is 
the axiomatization of iteraied reoisiori, a topic that has received considerable 
attention in recent years (see, for example, Boutilier, 1996; Darwiche and 
Pearl, 1997; Nayak et al., 2003; Rott, 2006). Extensions of the logic ILaginm 
that incorporate axioms for iterated revision have been recently investigated 
in (Zvesper, 2007). Finally, another line of research, which is pursued in 
(Bonanno, 2007b), deals with the conditions under which belief revision can 
be rationalized by a plausibility ordering on the set of states, in the sense 
that the set of states that are considered possible aftel' being informed that 
œ coincides with the most plausible states that are compatible with ¢. 
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Appendix 
Proof of the claim in Footnote 6, namely that axiom WND is equivalent to 
the following axiom: if é and w are Boolean, 

© 1(B¢A By A7AB=A¢) A Iġ > Bẹ. 
Derivation of WND from the above axiom ('PL' stands for 'Propositional 
Logic'): 
0-'(BdA By ^~nB~ag) = (Ip > By) above axiom, PL 


O07*(B¢A By A =~B=¢) > 
OU¢ > By) 1, rule RK for 0 


V= 


3. (Bọ^ By ^anBang) > 
OO (BéA BY AAB-¢) Temporal axiom 01 
4. (B¢A By AABRA¢) — Og —> By) 2,3, PL. 


Derivation of the above axiom from WND: 
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Od¢é- By 
2. ~O (I¢ > By) 
(Bb A By N ~nB=¢ 
3. O-1-'9 (1g By) 
> O-1-(Bbl By T. aBad 
4. O7'(Boél BYT. =B-¢) 
>01 O (Id By 
5. ~=(Iẹ > By) > 
O-'07U¢ > By 
6. 0-10 (Tọ > By) 3 (Id > Bu 
7. O71 Be r. By. aBad) > 
(I$ > Bo 


(Bor. By AABR7¢) > 
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Axiom WND 
1, PL 

2, rule RK for O-' 

3, PL, definition of O-!1 


Temporal axiom O2 
5, PL, definition of O-' and © 


4,6, PL. 
Q.E.D. 


PTOof of the claim in Fooinote 7, namely that axiom WN A is equivalent to 
the foliowing axiom: if ¢ and % are Boolean, 


©-1(B¢A By) AI 


¢— ~By. 


Derivation of WNA from the above axiom: 


O-'( Bor. =BY) T. Ig > ~By% 
OTIB T. “By) > (Ip > =B) 


O07 (BeT. =BY) > OU > =B4) 
(Bor. =BY) > O0! (Ber. =B4) 


(BeN-By) > O(g > ~By) 


above axiom 

1, PL 

2, rule RK for O 
Temporal axiom 01 
3,4, PL. 


Derivation of the above axiom from WNA: 


(B6\ -Bd) = Olg > =B4) 


2. 30 (16 ABy) > 
4(BéN-ABy 
3. O-1-,0 (1¢—-=By) 
=> OQ '7A(BéN ABy 
4. 0 (BbA=Bp) > 
0-19 (Tọ 3 =By 
5. -(1¢— ~By) > 
OORU > “By 
6. 0-10 (Tọ 5 By) > 
(I > =By 
7. 071(BéA7BY) = (Id > ~By 
8 © '(Bdl. ~By) lr. Iġ > ~By 


Axiom WNA 
1, PL 

2, rule RK for O-' 

3, PL and definition of O-' 
Temporal axiom O2 

5, PL, definition of O-' and © 
4,6, PL 


7, PL. 
Q.E.D. 
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Proof of Proposiiion 4.1. (1) Fix a frame that satisfies Pwno. an arbitrary 
model based on it and arbitrary &œ E Q, t; E Tand Boolean formulas ¢ and 
w and suppose that (a,t;) | (Bọ A By A AB-¢). Sinee (a,t,) = AB7d, 
there exists an w EBt, (a) such that (w,t,) = ¢. Thus Bz (a) # O. Fix an 
arbitrary t-: E T sueh that t+: — t-: and suppose that (a,f2) = I. Then 
Tr(a) = [Pla Fix an arbitrary l3 E B, (a). Sinee (a,t1) =| Bọ, (8,t1) H 
$. Sinee ¢ is Boolean, by Remark 2.2 (j3,12) | ¢. Henee l3 E Te (@). 
Thus Bz, (a) < Z;,(a). Hence, by Pwno. Bi, (a) < B, (a). Fix an arbitrary 
wE Ba (a). Then w EBt, (œ) and, sinee (a,t1) | By, (w,t1) H %. Since ' 
is Boolean, by Remark 2.2 (W,f2) = %4. Thus (a,t2) E By. 

Conversely, suppose that Pwno is violated. Then there exist œ E Q and 
ty,t: ET sueh that t: > 12, Bt (a) # O, Br (a) C Tae (œ) and Ba) £ 
B, (œ). Let pand q be atomie propositions and construct a model where 
Ipll = Z,,(@) x Tand Ilqll = @.,(a) x T. Then (a,t1) |} (Bp ABq ^,B,p). 
By hypothesis, there exists a 13 E B,,(a) sueh that 13 ¢ Brt (a), so that 
(13, 12) # q. Henee (a, 12) # Bq while (a, 12) H Ip, so that (a,12) # Ip > Bg. 
Thus, sinee +: ~> t2, WND is falsified at (a,t1). 

(2) Fix a frame that satisfies Pwna, an arbitrary model based on it and 
arbitrary œ E Q, t; E Tand Boolean formulas ¢ and % and suppose that 
(a,t,) EK B6A-By. Then there exists a 13 EBt, (a) sueh that (13, t1) H av. 
Fix an arbitrary t-: ET sueh that +: > +: and suppose that (a,t2) = Tẹ. 
Then Z;,(a@) = []rz’ Fix an arbitrary w E Bi,(a). Sinee (a,t,) E Bd, 
(w, tı) | œ. Sinee ¢ is Boolean, by Remark 2.2 (w, t2) = ¢ and therefore 
wE 7,,(a). Thus B, (a) < Ta (a) and, by Pwna. Ba (a) < Be (œ). Sinee 
(13, t1) H 7% and ~y is Boolean (because ‘ is), by Remark 2.2 (13,12) H ~. 
Sinee 13 EBt, (œ) and Bt (œ) C Brza), 13 E By2(a) and therefore (a, 12) H 
~By. 

Conversely, suppose that Pwna is violated. Then there exist a E Q 
and t7,12 ET sueh that +: > t: and Ba (a) C Z,(a) and Bi, (a) ¢ 
Ba (a). Let pand q be atomie propositions and construct a model where 
Ipll = Z,(a@) x Tand Ilqll = B,,(a) x T. Then (a,t1) EF Bp \,Bq and 
(a,t2) H Lpr.Bq, so that, sinee t-: > 12, (a,t1) FE ; O (lp >,Bq). QED. 


Proof of Proposiiion 4.3. First of all, note that, sinee O-' is a normal op- 
erator, the following is a theorem of ILa (henee of ILw): 


Ox A OTE > OTIA E). (1) 
It follows from (1) and axiom BU that the following is a theorem of ILa: 
OASTE OTA E): (2) 


Figure 3 below is a syntaetie derivation of WNC. Q.E.D. 
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0-1(B¢\ =B-¢) A 0-1BY > 


© 1(B¢\ AB7¢/\ By) Theorem of La 


0-1(BdN=B-¢ A By) NI¢ > Bo 


0-1(BéN ABad) N 0-1By A Ie > By 

I¢N\O-'(Bé\=B-¢) > (0-1BYy > By) 
0HB N =B=4¢) NOT -BY = 
0-1(B¢éN\ AB=¢\ =B4) 


—1-(BéN =By) = 


(Bd N ~By) > -(Bé \ AB N AB) 


(see (2) above) 
Equivalent to WND 
(see Footnote 6) 
1,2, PL 

3,PL 


Theorem of La 
(see (1) above) 
Tautology 
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O 

8. 0-1(B¢A=B=d ABY) > 0-1(Bd N~BY) 
a 
0- 


O71 (Be A =Ba¢\ ~By) 6, mie RK for O-' 


1(Bd A =Bag) A OQ-InBYy => 


7, PL, def. of O-! 


O-'(B¢A-=By) 5,8, PL 


BéN AB) NI¢ > -By 


11. 0- {Bẹ A -B=) NOBYA Ip > ~By 
12. I6 NO (Be A AB>¢) — (O71-By = ~By) 
13. (Q71ABYy > ~By) > (By > 0-1B4) 


14. I¢ h0 !(Bo N~B-¢) > (By > 0-'By) 
15. Ip N07} (BoA AB-¢d) > (By = 0-1B4) 


FIGURE 3. 


equivalent to WNA 
(see Footnote 7) 
9,10, PL 

11, PL 

tautology and 
definition of O-1 
12, 13, PL 

4,14, PL. 


PTOof of Pmposition 5.1. (1) Fix a frame that satisfies PND, an arbitrary 
model based on it and arbitrary & E Q, +: E Tand Boolean formulas ¢ and 


w and suppose that (a, tı) 


t: > t-: and (a, t2) 
exists a 1} E Bel(a 


H ABAdA By. Fix an arbitrary t-: ET sueh that 
H Iġ. Then Z;,(a) = [olez Sinee (a, t1) = ~B-¢, there 
sueh that (l3, t1) = ¢. Sinee ¢ is Boolean, by Remark 


2.2 (j3,t2) =| ġ and, therefore, 13 E Ta, (aœ). Thus By, (a) OTe (œ) #0 and, 
by PND, B, (œ) C Bg (œ). Fix an arbitrary w E B, (a). Then w E Byla) 


and, sinee (a,t,) = By, (w, t1) 


(w, t2) =| %. Henee 


(&œ,t2) = By. 


H w. Since 4% is Boolean, by Remark 2.2, 


Conversely, fix a frame that does not satisfy PND. Then there exist 
a EQ and t1,t2 ET such that t: >> 12, Byla) n Te (a) #0 and Be (a) É 


ByI(a). 


Let pand q be atomie propositions and construct a model where 
Ilpl = Bzj(c) x Tand Ill = Ze (œ) x T. Then (a,t,) 


FE ,B,q/ Bp and 


(a, 12) = Ig. By hypothesis there exists a 3 E ByXc) sueh that I ¢ Byla). 
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Thus (8,12) F pand therefore (a, 12) |: -,-Bp. Henee (a,t1) H 7 0 (Iq > 
Bp). 

(2) Fix a frame that satisfies Pya, an arbitrary model based on it and 
arbitrary @ E Q, +: E Tand Boolean formulas ¢@ and % and suppose that 
(aœ tı) = ABa(é A ~y). Fix an arbitrary +: E T sueh that t: — t: and 
suppose that (œ, t2) = Id. Then 142(a)= [¢],,. Sinee (œ, t1) F ~B>(¢A 
=p), there exists a 3 E By](a) sueh that (8,t;) = dA 7. Sinee ¢ and 4% 
are Boolean, by Remark 2.2 (;3,12) | dA -7w. Thus;3 E 172(a) and, by 
Pna,:3 E Bela). Thus, sinee (53,12) | 77, (a,t2) | ABy. 

Conversely, fix a frame that does not satisfy Pya. Then there exist 
a E Q and t7,t2 ET sueh that +: > t: and Ba (a) n1.t2(a) É Bi, (a). 
Let pand q be atomie propositions and construct a model where Ilpll = 
142(a) x Tand Ilgll = Ba (œ) x T. Then (a, 12) = lp A Bq and, therefore, 
(lati) Ea O (ip > -,Bq). Sinee Br (a) N 142(a) É Ba, (œ) there exists a 
BE e., (a) N 142(a) sueh that 8 ¢ B, (a). Thus (6,t1) H pA-'g. Henee 
(a,t;) E -,B-,(pA-q), so that axiom NA is falsified at (a, ¢,). 


(3) Fix a frame that satisfies Poa, an arbitrary model based on it and 
arbitrary œ EQ, t: E Tand Boolean formula ¢ and suppose that (a,t,) = 
—B~—gġ. Then there exists a B E Byj(a) sueh that (8,tı) = ¢. Fix an 
arbitrary t2 sueh that f; — t2 and suppose that (a,t2) |= [¢. Then 142@) 
[al rc! Sinee ¢ is Boolean and (f,t1) | ¢, by Remark 2.2 (8,12)  ¢. Thus 
BE 142(a) and, therefore, B, (a) 1.42(a) A O. By Poa. Bi, (a) < 142(a). 
Thus (a,t2) = Bd. Henee (a, t1) = OU¢ > Be). 

Conversely, suppose that Poa is violated. Then there exist œ E Q and 
t;,t-: ET sueh that t: — 12, Byla) nN 142(a) A 0 and By, (a) Z 142(c). Let 
p be an atomie proposition and construct a model where Ilpll = 142(a) x T. 
Then (a,t,) = -,B-,p and (a,t2) = Ip. By hypothesis, there exists a 8 E 
ByAc) sueh that 8 € 14a). Thus (3,12) F pand therefore (a, 12) - -,Bp. 
Henee (a, tı) ¥ O(Ip — Bp). Q.E.D. 


Proof of the claim in Remark 5.4, namely that WNA is a theorem of the 
logie ILa + NA: 


=B(¢ > 4) — OU => ~By) Axiom NA (see Footnote 8) 
B(¢é— 4) — (Bọ > By) Axiom K for B 

(BéN Bp) > =B(¢ > 4) 2, PL 

(BéN>Bp) — 0(1¢6 — =By) 1,3, PL. 


KRWN > 


Q.E.D. 


Proof of Proposiiion 6.1. (1) The proof of this part is straightforward and 
is omitted. 


(2) Fix a frame that satisfies property Px7. Let @ and t-: be sueh that 
(ati) | OU (dA Y) A BX), where ¢, and X are Boolean formulas. Then 
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there exists a t3 sueh that t: >> t3 and (a,t3) = I($ A w) A BX. Thus 
Teala) = fo Al t3' Fix an arbitrary t2 sueh that t 7 > t2 and suppose that 
(a,t2) EK Id. Then Tela) = [dl rz’ Sinee ġ and % are Boolean, by Remark 
22, [ØA bln = IPAP: Thus, sinee [PA Vs, < [liq To (0) < Tala). 
Henee by P g7. %,(a) N Bata ) < Ba (&). Fix an ebay 13 E Ba (a). If 
(j3,t2) E =p A Y) then (j3,t2) E (6A 4) > X. If (73,12) E o A 4, then, by 
Remark 2.2, (13,13) = A% and, therefore, 13 E Ti, (œ). Henee 13 E By3(a). 
Sinee (a,13) H| BX, (1313) | X and, therefore, (1313) E (e ^ 4) > 
Sinee (6A w) — X is Boolean (beeause ¢, ~ and X are), by Remark 2.2, 
(j3,t2) EF (6A) => X- Thus, sinee 3 E B,,(c) was ehosen arbitrarily, 
(a,t2) = BUPA 4) > X) 

Conversely, suppose that P g7 is violated. Then there exist t-i, t2, t3 and 
a sueh that +: > 12, t: >> t3, Ta (a) C Te (a) and Zu (a) O Be (a ) g “en 
Let p, q and r be atomie propositions and construct a model where Hp 
Taela) xT, llall = T,,(a) xT and IITll = B,,(@) x7. Then, (a, 13) = Br and, 
sinee Za (aœ) < Tala), Tala) = ipAglt3 so that (a,tz) FE L(p^q). Thus, 
sinee t: > 13, (a, z H Bae \q)\ @-, Byeonstruetion, (a,12) = Ip. 
Sinee a )NB,, (a) É Besla), there exists a 13 E Ta (œ) N Ba (œ) sueh that 
13 ¢ By3(a). Thus (13, t2) E - 7 ARERO sinee 13 E Za, (a), (13, ta) | p^q 
and, by ee 2.2, (13 to) = pAgq. Thus, (13, t2) É (pN gq) — r. Sinee 
13 E B,2(a) it follows that (œ, t2) Æ B ((@Nq)—r). Hence, sinee t-: — 12, 
(a,t,) F O(lp —> B((p Nq) T)) so that axiom K7 is falsified at (a,t1). 


(3) Fix a frame that satisfies property Pgs. Let ¢, p and X be Boolean 
formulas and let a and t ; be sueh that (æ, t1) EF O(T@A4B-(dAWP)AB() > 
X)). Then there exists a t; sueh that +: > t: and (a,12) = Tẹ A -,B-,(ġ A 
p) A Bp > X). Thus Za (œ) = [¢],, and there exists a 13 E Ba, (œ) sueh 
that (13,12) | @A w. Fix an arbitrary 13 sueh that +: > t3 and suppose 
that (a,t3) Æ I($ A 4%). Then Ta (&) = [¢ Awl 13’ Sinee ġ A% is a Boolean 
formula and (13. t2) = 6A 4%, by Remark 2.2, (13, t3) = $A% and therefore 
13 E Ta (a). Henee Ta (œ) N Bla) Æ 0. Furthermore, sinee ¢ is Boolean, 
by Remark 2.2, fel B = fel rz’ Thus, sinee E Avl B< [al t3 it follows that 
Ti,(a) < Z;,(a). Hence, by property Pgs, Bela) < Tala) By, (a). Fix an 
arbitrary y E Ba (œ). Then y E Z;,(a) N Be (&) and, sinee (a, t2) | Bia > 
X), (%,t2) Eb —> X- Since 7 — X is a Boolean formula, by Remark 2.2 
(yts) =E Y > X. Sinee y E Tu (a) and Tala) = [GA Y] (mta) E Y. 
Thus (7, 3) H| X- Henee (&,t3) H BX- 

Conversely, fix a frame that does not satisfy property Pxs. Then there 
exist t ],t2,t3 and œ sueh o t] = t2, t1 > t3, Ta l@) N Ba (a) Æ O, 
Tala) < Ta (@) and By, (a ) g T,,(a) N Be (a). Let » qand r be atomie 
propositions and construct a model where | oll - Tala) xT, lial- Z., (œ) x 
Tand IITIl = (Z, (œ) n B2(a)) x T. Then (a,t2) | Ip and, sinee Ty (a) C 
Tela), if w E Ta (œ) then (w,t) - p Aq for every t E T. Thus, sinee 


SS ee 
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Ti, (a) NB, (a) £0, (a, te) H ,B,(p^q). Fix an arbitrary wE B, (a); if 
WE Z, (a) then (w,t2)| r; ifw ÉT, (œ) then (w,t2) |= ,q; in either case 
(w,t2) = q> T. Thus (a,t2) | B(q > r). Hence (a,t2) F lp\,B,(pN 
q) \ B(q —> T) and thus (a,t7) = © (Ip A, B,(p Nq)^N B(q > T)). Since 
T,,(a) = lale and Ta (aœ) = fp] t, and, by Remark 2.2, [Pla = [Pla and 
Tala) < Ta, (a), if foliows that Ze (a) = [pA qlt 3 so that (a, t3) = I(pAq). 
Since Brxa) É 113(&œ) N By2(a), there exists a BE Br3xa) such that l} ¢ 
Tala) NB, (a). Then (13, t3) | ar and therefore (a,t3) = ,BT. Thus 
(a,t3) F I(p Nq) > BT and hence, (a,t1) F O(I(p Aq) — BT), so that 
axiom K8 is falsified at (a,t,). 


(4) Let ¢ be a Boolean formula, œ EQ, t E Tand suppose that (a,t) = 
I¢ \ aA. Then Ti(&œ) = [¢], and there exist l E Q that (j3,t) H ¢. 
Thus Z;(a@) # 0 and, by the above property, B,(a) Æ O. Fix an arbitrary 
formula w and suppose that (a,t) | By. Then, Vw E Bila), (wt) H 4%. 
Since B,(a) Æ O, there exists a y E B,(a). Thus (y,t) H w and hence 
(a,t) = AB. 

Conversely, fix a frame that does not satisfy property Pwc. Then there 
exist œ EQ and t ET such that 7,(a) #4 0 while B,(a@) = O. Let p be an 
atomie proposition and construct a model where Ilpll = Z,(œ) x T. Then 
(a,t) = lp. Furthermore, since Z,(a) 4 O, there exists a BE T.(a). Thus 
(13, t) | pand hence (a,t) KE ,A,p. Since B;(a) = O, (a,t) H} By for every 
forrnula b, so that (a,t) = Bp/\B,p. Thus WC is falsified at (a,t). QED. 


Proof of the claim in Fooinote 11. (BR3! ==> BR3). Fix arbitrary E, F E 
E such that E C F and RF nE #0. Then EnF = E, so that (EnF) E E 
and REnF - RE. Thus, by (BR3’), RE - RF n E. 

(BR3 + BRI ==> BR3!). Let E, F € E be such that (EN F) E £ and 
RF nE #0. By (BRI), RF < F so that RF n F = RF. Hence 


RF n (En F) = RF NE. (t) 


Thus RFn(EnF) #0. Hence, since EnF < F, it foliows from (BR3) 
that REnF = RF Nn (En F). Thus, by (t), REnF= RF nE. Q.E.D. 


In order to prove Proposition 7.4 we need the foliowing lemma. We shali 
throughout denote the complement of a set E by 7E. 


Lemma A.I. Let (Q,E£,O,R, V) be a rieh belief revision model. Then, for 
every formula ¢ E ®o, Iọ|| = 0 if and only if ¢ is a contradiction (that is, 
~g is a tautology). 


Proo]. If ¢ is a tautology then ||¢|| = Q. If ¢ is a contradiction then ~g 
is a tautology and thus ||7¢]| = "||¢|| = Q, so that ||ọ|| = o. If ¢ is nei- 
ther a tautology nor a contradiction then it is equivalent to a formula of 
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the form (Via Nie Qij)) where eaeh Qij is either an atomie proposition 

or the negation of a atomie proposition (see Hamilton, 1987, p. 17, Corol- 

lary 1.20). By definition of rieh model, for every formula Nia Qij, there 

is a state wi sueh that wi l= Njai O.; Thus ||¢]| = [V (Aja Qij)ll = 
n m se 

LE a lee Qij ll 2 {WL wee Wn } fp O. Q.E.D. 


PTOof of Proposiiion 74. Let (Q,€,0,R, V) be a eomprehensive belief re- 
vision model and define K = {4% E ®o : Ro C ||p)||}. First we show that K 
is deduetively closed, that is, K = ar, If Y E K then %4 E Fleer be- 
eause p — y) 1s a tautology; thus K C LK IPL. To show that [KIPL C K, let 
pE [AK], that is, there exist ¢,,...,¢, E K sueh that (¢, l\.... N. dp) > Y 
is a tautology. Sinee I1¢,N\.... \. dy = |l N... NI, 1, and ¢; E K (that 
is, Ro C ||¢;||) for all i = 1,...,”, it follows that Ro C ||¢, 1... N nll- 
Sinee (¢, N... Np) > Y% is a tautology, I(@, N... \.d,,) > wl = Q, that 
$, 1... pall C ||P]. Thus Rola) C |||, that is, Y E K. Next we show 
that ier Æ Po (eonsisteney). By definition of one-stage revision frame 
(see Definition 7.2), © Æ O; thus, by property BR2, Ro Æ O. Choose an 
arbitrary atomie proposition p E S. Then I(p\. =p) = 0 and therefore 
Ro Z I(ph.,p)1l, so that (pll.ap) ¢ K. Sinee K = [K]°™, (pli. ap) ¢ [K]?™. 
Next we show that AGM postulates (®1)-(@5a) and (@6)-(@8) are satis- 
fied. For every formula ¢ E ®o, define K®(¢) = {4 E ®o : Rygy C lell} 
(note that, sinee the model is eomprehensive, for every ¢ E ®o, ||ġ|| E £). 

(®1) Fix an arbitrary ¢ E Po. We need to show that {7 E ®o : Rig E Ilym 
is deduetively closed. We omit this proof sinee it is a repetition of the 


is, 


argument given above for K. 


(@2) Fix an arbitrary ¢ E Po. We need to show that ¢ E K(¢), that is, 
that Ry & Il]. This is an immediate eonsequenee of property BR1 of 
Definition 7.2. 


(@3) Fix an arbitrary ¢ E Po. We need to show that K®(¢) <[KU{@}]P4, 
Let y E K©(¢), that is, Ry E |||]. First we show that (¢ > 4) E K, 
that is, Ro C || > || = (lellu lle]. If Ro C I] there is nothing to 
prove. Suppose therefore that Ro N \|l| # O. Then, by property BR4 of 
Definition 7.2, 


Rio = Ron lell- 8) 


Fix an arbitrary w E Ro. If w ¢ 14|| then w E \|>¢|| and thus WEil¢ > wit 
if wE |All, then, by (3), w E Ry | and thus, sinee Ry¢| € ale wE ale so 
that w E ||¢ > *||. Hence (¢ > Y) E K. It follows that Y E |K U{@}]P. 


(@4) Fix an arbitrary ¢ E o. We need to show that if = ¢ K then 
[K U {o}]P" < K°(g). Suppose that =¢ ¢ K, that is, Ro Z ||>¢I| = “lll, 
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that is, Ro N ||¢|| 4 0. Then by property BR4 of Definition 7.2, 
Ryo = RoN liell. (4) 


Let XE |K U{¢}]?", that is, there exist ¢,,...,¢, E K U{d} sueh that 
(¢,A...N¢,,) > Xis a tautology. We want to show that XE K®(¢), that 
is, Ry C IIxll. Sinee (¢, A... A $n) > Xis a tautology, ||(¢1 A- An) > 
xii = Q, that is, I(¢, A... A a)l C IIxll- If ¢; E K for every i = 1,...,7, 
then Ro < ||(¢; A---A n)l| and thus Ro < IIxll- Hence, by (4), Ryo < 
lIxll. If, for some j 1,...47, db; ¢ K, then we ean assume (renumbering 
the formulas, if neeessary) that ¢, E K, for every i = 1,...,n - 1, and 
n Z K, whieh implies (sinee ġ; E K U{¢} for all i = 1,...,n) that 
Pn = $. Since, by hypothesis, CA A---A¢,,\¢ — Xis a tautology and, 
furthermore, it is propositionally equivalent to (¢,A:..A¢,_,;) > ($ > X), 
[A Agna) > (6 > gi = Q, that is, |]. A---Ab,—all E ll > xii, so 
that, sinee Ro C 1¢,A-..A¢,,_1|] (beeause ¢),...,¢,_, EK), Ro C Id > 
xii: Thus Ron llel] < |l¢ll n || > xii < IIxll- Hence, by (4), Rye < IIxll- 


(®5a) If ¢ is a eontradietion, ||¢|| = 0. By property BR1, Rjg} < ||¢]. 
Henee Rj) = 0 and, therefore, K°(d = {4 E Bo : Rygy E lvl} = Bo. 


(@6) If => Y% is a tautology then Ido wll = Q, that is, |||] = ||1l, Henee 
Risi = Ryy| and thus K($ = {X E Bo : Rygy < Iixll} = /X E ®o : 
Riy] < lx} = K? (4). 


(®7) Fix arbitrary ġ, p E Po. We need to show that KE(gpAy C [K©(¢)U 
{p}]P4. Let X E KE(¢ Ay), that is, 


Ri garwi| E Ilx. (5) 


First we show that Rj C IA% > xi = “lle Ayl Ullxii- If Rel S 
||6A%|| there is nothing to prove. Suppose therefore that Ry) n||PA vl] A 
O. Then, by property (BR3) (with E = ||6A || and F = |ẹll), 


Ryg AGA tll = Rygayy- (6) 


Fix an arbitrary w E Rig). If w é || A | then w E ||-(¢A %)]] and 
thus w E ||(¢A% —> v]: if w E || A %], then by (5) and (6), WEIIxii 
so that w E |(¢ ^% — xii Henee Rjg} © (6A % — xii, that is, 
(oA % — X) E K®(¢). Sinee (¢ A% — X) is tautologieally equivalent 
to (b — (¢ — X)), and, by (@1) (proved above), K@(¢ is deduetively 
closed, (y — (¢ > X)) E K®(¢). Furthermore, by (@2) ¢ E K®(¢). Thus 
fw, (Y > (6 > x)), 6} C K°(d Uf} and therefore XE [K@(d Ufp}]P. 


(@8) Fix arbitrary œ, E Po. We need to show that if ~ ¢ K®(¢ 
then [K®(¢ Ufy” C KE( Ny). Suppose that ~p ¢ KE (¢), that is, 
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Ris Z “ll = Wl, ie, Riggin lell A 0. Then by property (BR3/) (see 
Footnote 11), 


Rigal = Riel n ll. (7) 


Let XE [K®(¢) Up}, that is, there exist $,,... ¢, E K®(¢) U {4} 
such that (¢,A...A¢,) > Xis a tautology. We want to show that 
XE KE( Ny), that is, Rigal E [xll- Since (¢,A-- An) > Xis a 
tautology, ($A Ada) > xii = Q, that is, ||(¢1 ^ Ad, C xli- 
If ¢, E K($) for every i = 17,...,N, then Rye E I A Agn) 
and thus Rijs] < lixll- Hence, by (7), Ry gayi < IIxil. If, for some 
j= 1,...n, 6; € KE(¢), then we can assume (renumbering the for- 
mulas, if necessary) that $, E k®(¢), for every i = 17,...,n -1, and 
bn € K©(), which implies (since ; E K®(d) U {4} for all i = 1,...,n) 
that ¢, = . Since, by hypothesis, (¢,\...\¢,_; A4) > Xis a tau- 
tology and it is propositionally equivalent to (i A A Pn) > (4 => X), 
IA Agni) > (>HI = Q, that is, (81^ Anall E Ile > 
Xii, so that, since Ry | < I(o. A+++ A dns) Il (because Die ca Pn E 
KE($)) Riej < lY > xii- Thus Riel n Wl < lvl n ly > xii < IIxll- 
Hence, by (7), Ryjgayi < Ixl. 


Next we show that, if the model is rich, then (®5b) is satisfied. 

(®5b) If the model is rich and h is not a contradiction, then by Lemma 
A.1 ||¢|| Æ O. Thus, by property BR2, Ry) 7 O. Fix an arbitrary p E S. 
Since lipA>pl| = 0, it follows that Rjg] É lip\—p|| and therefore (p~p) ¢ 
K®(¢). Since, by (@1) (proved above), K®(d) = [K@(¢)|°", it follows that 
[KE] # Bo. QED. 


Before proving Proposition 7.5 we note the following. 


Definition A.2. A set E C 2? of events is called an algebm if it satisfies 
the following properties: (1) Q E E, (2) if E E E then 'E E E and (3) if 
E,F E [then (EUF) E €.% 


Remark A.3. In a belief revision frame where ÈE is an algebra, property 


(BR3/) (see Footnote 11) is equivalent to: YE, FE E, 
if RF NEF 0 then R gnf = RFNE. (BR3//) 


PTOof of Pmposition 7.5. Let M be the set of maximally consistent sets 
(MeS) of formulas for a propositional logic whose set of formulas is o. 
For any F C Po let Me = {wE M : F C w}. By Lindenbaum's lemma, 


15 Note that from (1) and (2) it follows that 0 E € and from (2) and (3) it follows that 
if E, PEE then (E n P) E E. In fact., from E, PEE we get, by (2), TE, (F EE and 
thus, by (3), (EUTF) E E; using (2) again we get that ("HU'F) = (En P) EE. 
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Mp £0 if and only if F is a consistent set, that is, |[F]P}  ®o. To simplify 
the notation, for œ E Po we write Mg rather than M{4}. 

Define the following belief revision frame: Q = M , E = {My : ¢ E ®o}, 
O= Q, Ro= Mx and, for every ¢ E Bo, 


D if ġ is a contradiction 
Rm Me OMx if ¢ is consistent and Mg n M K £ 0 


ly 
Mx 2($) if ¢ is consistent and Mọ n Mx = O. 


First of all, note that € is an algebra. (1) M € E since M = Mypy—p) 
where p is any atomie proposition. (2) Let ¢ E o. Then Mg E E and 
|My|| = {w EM: ¢ € w}. By definition of MCS, for every w € M, 6 € w 
if and only if > Ew. Thus “|Mg|| = My E E. (3) Let ġ, Y E o. Then 
My My E E and, by definition of MCS, Mg U My = Mọwy E E. 

Next we show that the frame so defined is indeed a one-stage revision 
frame, that is, it satisfies properties (BR1)-(BR4) of Definition 7.2. 


(BR1) We need to show that, for every #6 E o, Ru, C Meo. If ¢ is a 
contradiction, then My = o0 and, by construction, Ru, =o. If dis 
consistent and Mg n Mx # 0 then Ry, = Mgn Mx C My. If ¢ is 
consistent and Mg n MK = 0 then Ru, = Mx @(4)- Now, if io’ E M0 4) 
then K®(¢) C io’ and, since by AGM postulate (@2), ¢ E K®(), it follows 
that ¢ Eco’, that is, w: E Mg. Hence Mx 04) C Mg. 


(BR2) We need to show that, for every PE ®o, if Mg A0 then Rm, AO. 
Now, My #0 if and only if ¢ is a consistent formula, in which case either 
Ru, = Men MK ifMgn MK #0 or Ru, = Mke) if Mgn MK = 0. 
In the latter case, by AGM postulate (@5b), K©(@) is a consistent set and 
therefore, by Lindenbaum's lemma, Mx 0 (4) Æ O. 


(BR3) Instead of proving (BR3) we prove the equivalent (BR3I!) (see Re- 
mark A.3 and footnote 11), that is, we show that, for every ġ,% E ®o, if 
Rm, OM, #0 then Ruynmy = Ru, My. First note that, by definition of 
MCS, Mgn My = Méry. Since Rm, 4 O, ¢ is a consistent formula and ei- 
ther Rms = MNM k, ifMgNM x # O, or Rat, = Mx 04); if MgNM x = 0. 
Suppose first that Mọ n MK # 0. Then Rm, n My = Mg n M K n My = 
Meéav n MK # O, and, thus, by construction, Ry,,, = Mgryn M K. Thus 
Rmomy = Rmsay = Ru, n My. Suppose now that Mgn M K = O. Then, 
by construction, Rm, = Mko(g) and, since Mg,ynMx = Mg OMynMx © 
Men MK = 0 we also have that Ru,,,, = Mxegry). Thus we need to 
show that Mxegay) = Mog) My. By hypothesis, Rm, My # O, that 
is, Mx og) My #0. This implies that ~~ ¢ K®(@).1° Hence, by AGM 


16 Suppose that =~ E K®(é). Then, for every w E Mio) w D K®(¢) and, therefore, 
=Y E w. But this implies that Mgeçg) N My = O. 
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postulates (®7) and (®8), 


[KPH U {y " = KPa). (8) 


Let w E Myecgayy. Then KË(¢ A y) < wand, since K®(¢) U {4} < 
IKE UH”, it foliows from (8) that K®(¢) U {4} < w. Thus w E 
Mxo(g) NMMy. Conversely, let w EM og) My. Then K®(¢) U{Y} < v. 
Hence, by definition of MCS, [K®(¢) UL hPL C w. It foliows from (8) that 
K®(¢A%) < w, that is, wE Mx ogay). Thus Mregay) = Mog) n My, 
that is, Reynmy = Ru, n My. 

(BR4) Since O = Q and, by construction, Ro = Mx, we need to show that, 
for every formula ¢, if Mọ n m x #0 then Ry, = Men m x. But this is 
true by construction. 


Consider now the model based on this frame given by the foliowing 
valuation: for every atomie proposition pand for every WE Q, w = p if and 
only if p E w. It is weli-known that in this model, for every formula ¢,! 


lell = Mg. (9) 


Note also the foliowing (see Chelias, 1984, Theorem 2.20, p. 57): YF C 
Po, Vo E Po, 


o E[F]PH if and only if GEw, Yw E Mp. (10) 


We want to show that (1) K = {4 E o : Ro C ||%||} and, (2) for every 
PE o, KE (¢) = {PE Bo : Rygy E liell} 
(1) By construction, O = Q and Re = Mx and, by (9), for every forrnula 5, 
||| = My. Thus we need to show that, for every formula %, ù E K if and 
only if Mk C My. Let y E K and fix an arbitrary w E Mx. Then K Cw 
and thus % E w, so that w € My. Conversely, suppose that Mk C My. 
Then % E w, for every w E Mx. Thus, by (10), y E [|K]P}. By AGM 
postulate (@1), K = [K]P4. Hence PEK. 


(2) Fix an arbitrary formula ¢. First we show that K®(¢) C {4 E ®o : 
Rm, C lvl}. Let y E K°(¢). If ¢ is a contradiction, Rm, = o and 
there is nothing to prove. If ¢ is consistent then two cases are possible: 


17 The proof is by induction on the complexity of d. If = p, for some sentence letter p, 
then the statement is true by construction. Now suppose that the statement is true of 
$1, 2 E ®o; we want to show that it is true for ~} and for (6, V 2). By definition, 
w |= 7, if and only if w ¥ @, if and only if (by the induction hypothesis) ¢, ¢ w if 
and only if, by definition of MCS, ~} E w. By definition, w = (4, V $9) if and only 
if either w | 4, in which case, by the induction hypothesis, 6; E w, or w F $9, in 
which case, by the induction hypothesis, œo E w. By definition of MCS, (4, Vés) Ew 
if and only if either ¢; Ew or $9 Ew. 
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© MgnMx = 0 and Gi) Mg nmk #0. Incase (i) Rm, = Mkeçg). 
Since, by hypothesis,  E K©(¢), Mog) E My and, by (9), My = Illl. 
Thus Ry, < |||. In case Gi), Ru, = Mọ N MK. First of all, note that 
MgOMx = Muy}. Secondly, it must be that —¢ ¢ K. '8 Hence, by AGM 
postulates (®3) and (@4), K®(¢) = [K U{@}]P4. Since, by hypothesis, 
pe K©(¢), Y B[KU{P}]P". Hence, by (10), Y E w, for every wE Mxuyg}. 
Thus Mxuie} < My. Hence, since Ru, = Mg n Mk = Mxuyg}, Ru, < 
My. Next we show that {yp E ®o : Ru, C |||} E K@(¢). Suppose that 
Ru, © ||| = My. If ¢ is a contradiction, then, by AGM postulate (5a), 
K®(é) = $o and, therefore, y E K®(¢). If d is not a contradiction, then 
either (i) Mg NM g = 0 or (ii) Mọ NM g FO. Incase (i) Rm, = Mx (4): 
Thus, since, by hypothesis, Ru, < My, we have that M keg) < My, 
that is, for every w E Mog), Y E w. By (10) p E [K®(¢)]P™ and, by 
AGM postulate (@1), [K®(¢)|P4 = K®(¢). Thus y E K®(¢). In case 
(ii), Ru, = Mz n Mx. Thus, since, by hypothesis, Riz, C My, we have 
that Mgn MK C My, from which it follows (sinee Mg n M K = Mxu{}) 
that Mku} © My. This means that, for every w E Myuyg}, V E vw. 
Hence, by (10), y E [K U{@}]P4. Since Mg ANMx 4 0, =¢ ¢ K and, 
therefore, by AGM postulates (@3) and (@4), K®(¢) = [K U{@}]P4. Thus 
We K®(¢). Q.E.D. 


Proof of Proposiiion 7.6. In view of Corollary 6.3 it is sufheient to show 
that (1) axiom CMP is characterized by property Pcwp and (2) Icon is 
characterized by seriality of /t. 

(1) Fix an arbitrary model based on a frame that satisfies property 
Pomp. Fix arbitrary œ E Q, tą E Tand Boolean formula ¢ and suppose 
that (œ, ta) = AAng. Let E = [¢]to. Then E #4 0. We want to show that 
(œ, to) H= OI. By property Pomp, there exists at E T such that ta >t 
and It(w) = E. Since ¢ is Boolean, [¢],, = [¢],. Thus (a,t) = Tọ and 
hence (a, to) E OTe. 

Conversely, fix a frame that violates property Pcivip. Then there exist 
a EQ, ta ET and E E2°\{@} such that, Vt ET, ifta — t then It(w) + E. 
Construct a model where, for some atomie proposition p, IIpll = E x T. 
Then, Vt ET with ta > t, (a,t) É lp. Thus (a,ta) F Olp. 

(2) Fix an arbitrary model based on a frame where Z, is serial and 
suppose that ,/(¢ /\ 4@) is not valid, that is, for some a@ E Q, t E Tand 
forrnula >, (a,t) = I(@A7¢). Then Fifa) = [6A 7¢],. But [6A 74], = O, 
while by seriality Z,(a@) Æ O, yielding a contradiction. 

Conversely, fix a frame where Z, is not serial, that is, there exist t E T 
and œ E Q such that J (œ) = O. Since, for every formula ¢, [$ A >¢]t = O, 
it follows that (a,t) = I(¢ Ang) so that ~I (¢ A~e) is not valid. Q.E.D. 


is [f= E K then 7@ E w for every w E Mg and therefore Mg n Mk = O. 
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Abstract 

We contrast Bonanno's ‘Belief Revision in a Temporal Framework' 
(Bonanno, 2008) with preference change and belief revision from the 
perspective of dynamic epistemic logic (DEL). For that, we extend the 
logic of communication and change of van Benthem et al. (2006b) with 
relational substitutions (van Benthem, 2004) for preference change, 
and show that this does not alter its properties. Next we move to a 
more constrained context where belief and knowledge can be defined 
from preferences (Grove, 1988; Board, 2002; Baltag and Smets, 2006, 
2008b), prove completeness of a very expressive logic of belief revision, 
and define a mechanism for updating belief revision models using a 
combination of action priority update (Baltag and Smets, 2008b) and 
preference substitution (van Benthem, 2004). 


1 Reconstructing AGM style belief revision 


Bonanno's paper offers a rational reconstruction of Alchourrén Gärdenfors 
Makinson style belief revision (AGM belief revision) (Alchourrén et al., 
1985; see also Gärdenfors, 1988 and Gärdenfors and Rott, 1995), in a frame- 
work where modalities B for single agent belief and Z for being informed 
are mixed with a next time operator O and its inverse O-'. 

Both the AGM framework and Bonanno's reconstruction of it do not 
explicitly represent the triggers that cause belief change in the first place. 
lp expresses that the agent is informed that y, but the communicative 
action that causes this change in information state is not represented. Also, 
y is restricted to purely propositional formulas. Another limitation that 
Bonanno's reconstruction shares with AGM is that it restricts attention to 
a single agent: changes of the beliefs of agents about the beliefs of other 
agents are not analyzed. Im these respects (Bonanno, 2008) is close to 
Dynamic Doxastic Logic (DDL), as developed by Segerberg (1995, 1999). 

AGM style belief revision was proposed more than twenty years ago, and 
has grown into a paradigm in its own right in artificial intelligence. In the 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 81-104. 
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meanwhile rieh frameworks of dynamic epistemic logic have emerged that 
are quite a bit more ambitious in their goals than AGM was when it was 
first proposed. AGM analyzes operations +% for expanding with y, —p 
for retracting y and *ọ for revising with y. It is formulated in a purely 
syntactic way, it hardly addresses issues of semantics, it does not propose 
sound and complete axiomatisations. It did shine in 1985, and it still shines 
now, but perhaps in a more modest way. 

Bonanno's paper creates a nice link between this style of belief revision 
and epistemicjdoxastic logic. While similar in spirit to Segerberg's work, 
it addresses the question of the rational reconstruction of AGM style belief 
revision more explicitly. This does add quite a lot to that framework: deal’ 
semantics, and a sound and complete axiomatisation. Still, it is fair to say 
that this rational reconstruction, nice as it is, also inherits the limitations 
of the original design. 


2 A broader perspective 


Meanwhile, epistemic logic has entered a different phase, with a new focus 
on the epistemic and doxastic effects of information updates such as public 
announcements (Plaza, 1989; Gerbrandy, 1999). Public announcements are 
interesting because they create common knowiedge, so the new focus on 
information updating fostered an interest in the evolution of multi-agent 
knowledge and belief under acts of communication. 

Public armouncement was generalized to updates with ‘action modeis' 
that can express a wide range of communications (private announcements, 
group announcements, secret sharing, lies, and so on) in (Baltag et al., 1998) 
and (Baltag and Moss, 2004). A further generalization to a complete logic 
of communication and change, with enriched actions that allow changing 
the facts of the world, was provided by van Benthem et al. (2006b). The 
textbook treatment of dynamic epistemic logic in (van Ditmarsch et al., 
2007) bears witness to the fact that this approach is by now weil established. 

The above systems of dynamic epistemic logic do provide an account of 
knowledge or belief update, but they do not analyse belief revision in the 
sense of AGM. Information updating in dynamic epistemic logic is mono- 
tonie: facts that are announced to an audience of agents cannot be unlearnt. 
Van Benthem (2004) calls this 'belief change under hard information’ or 
‘eliminative belief revision’. See also (van Ditmarseh, 2005) for refiection 
on the distinction between this and belief change under soft information. 

Assume a state of the world where p actually is the case, and where you 
know it, but I do not. Then public armouncement of p will have the effect 
that I get to know it, but also that you know that I know it, that I know 
that you know that I know it, in short, that p becomes common knowiedge. 
But if this announcement is followed by an announcement of -,p, the effect 
will be inconsistent knowledge states for both of us. 
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It is clear that AGM deals with belief revision of a different kind: 'be- 
lief change under soft information’ or 'non-eliminative belief revision’. In 
(van Benthem, 2004) it is sketched how this can be incorporated into dy- 
namic epistemic logic, and in the closely related (Baltag and Smets, 2008b) 
a theory of 'doxastic actions' is developed that can be seen as a further step 
in this direction. 

Belief revision under soft information can, as Van Benthem observes, be 
modelled as change in the belief accessibilities of a model. This is different 
from public announcement, which can be viewed as elimination of worlds 
while leaving the accessibilities untouched. 

Agent i believes that y in a given world w if it is the case that ọ is 
true in all worlds ¢ that are reachable from wand that are minimal for a 
suitable plausibility ordering relation <;. Inthe dynamic logic of belief revi- 
sion these accessibilities can get updated in various ways. An example from 
(Rott, 2006) that is discussed by van Benthem (2004) is {tA for so-called 
‘lexicographic upgrade': all A worlds get promoted past all non-A worlds, 
while within the A worlds and within the non-A worlds the preference re- 
lation stays as before. Clearly this relation upgrade has as effect that it 
creates belief in A. And the belief upgrade can be undone: a next update 
with ~A does not result in inconsistency. 

Van Benthem (2004) gives a complete dynamic logic of belief upgrade 
for the belief upgrade operation fA, and another one for a variation on 
it, TA, or ‘elite change’, that updates a plausibility ordering to a new one 
where the best A worlds get promoted past all other worlds, and for the 
rest the old ordering remains unchanged. 

This is Laken one step Iurt-her in a general logic Ior changing prefer- 
ences, in (van Benthem and Liu, 2004), where upgrade as relation change 
is handled for (reftexive and transitive) preference relations <;, by means 
of a variation on product update called product upgrade. The idea is to 
keep the domain, the valuation and the epistemic relations the same, but 
to reset the preferences by means of a substitution of new preorders for the 
preference relations. 

Treating knowledge as an equivalence and preference as a preorder, with- 
out constraining the way in which they relate, as is done by van Benthem 
and Liu (2004), has the advantage of generality (one does not have to spec- 
ify what ‘having a preference’ means), but it makes it harder to use the 
preference relation for modelling belief change. If one models ‘regret’ as 
preferring a situation that one knows to be false to the cutrent situation, 
then it follows that one can regret things one cannot even conceive. And us- 
ing the same preference relation for belief looks strange, for this would allow 
beliefs that are known to be false. Van Benthem (private communication) 
advised me not to lose sleep over such philosophical issues. If we follow that 
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advice, and call 'belief' what is true in all most preferred worlds, we can still 
take comfort from the fact that this view entails that one can never believe 
one is in a bad situation: the belief-accessible situations are by definition 
the best conceivable worlds. Anyhow, proceeding from the assumption that 
knowledge and preferenee are independent basic relations and then study- 
ing possible relations between them has turned out very fruitful: the recent 
theses by Girard (2008) and Liu (2008) are rieh sourees of insight in what 
a logical study of the interaction of knowledge and preferenee may reveal. 

Here we will explore two avenues, different from the above but related to 
it. First, we assume nothing at all about the relation between knowledge on 
one hand and preferenee on the other. We show that the dynamic logic of 
this (including updating with suitable finite update modeis) is complete and 
decidable: Theorem 3.1 gives an extension of the reducibility result for LCC, 
the general logic of communication and change proposed and investigated 
in (van Benthem et al., 2006b). 

Next, we move closer to the AGM perspective, by postulating a close 
conneetion between knowiedge, belief and preference. One takes prefer- 
enees as primary, and imposes minimal conditions to allow a definition of 
knowledge from preferences. The key to this is the simple observation in 
Theorem 4.1 that a preorder can be turned into an equivalence by taking 
its symmetrie closure if and only if it is weakly connected and conversely 
weakly connected. This means that by starting from weakly and converse 
weakly connected preorders one can interpret their symmetrie closures as 
knowledge relations, and use the preferences themselves to define conditional 
beliefs, in the well known way that was first proposed in (Boutilier, 1994) 
and (Halpern, 1997). The mulLi-agenL version of this kind of condit.ional 
belief was further explored in (van Benthem et al., 2006a) and in (Baltag 
and Smets, 2006, 2008b). We extend this to a complete logic of regular dox- 
astic programs for belief revision models (Theorem 4.3), useful for reasoning 
about common knowiedge, common conditional belief and their interaction. 
Finally, we make a formal proposal for a belief change mechanism by means 
of a combination of action model update in the style of (Baltag and Smets, 
2008b) and plausibility substitution in the style of (van Benthem and Liu, 
2004). 


3 Preference change tn Lee 


An epistemic preferenee model M for set of agents J is a tuple (W, V, R, P) 
where W is a non-empty set of worlds, V is a propositional valuation, Ris 
a function that maps each agent i to a relation R; (the epistemic relation 
for 1), and P is a function that maps each agent i to a preferenee relation 
Pi. There are no conditions at all on the R; and the Pi (just as there are 
no constraints on the R; relations in LCC (van Benthem et al., 2006b)). 
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We fix a PDL style language for talking about epistemic preference mod- 
els (assume p ranges over a set of basic propositions Prop and i over a set 
of agents J): 


4 Tip I-¥ Iyı A p2 Ifr]e 
tt ~i I>; |? | T1; T2 | m1 U Ta | 7* 


This is to be interpreted in the usual PDL marmer, with |r]M giving the 
relation that interprets relational expression z+ in M = (W, V,R, P), where 
~; is interpreted as the relation R; and >, as the relation Pi, and where 
the complex modalities are handled by the regular operations on relations. 
We employ the usual abbreviations: is shorthand for =T, p1 V yg is 
shorthand for -,(7~1 \7¢2), 1 > p2 is shorthand for -,(Y1 A2), G1 © Y2 
is shorthand for (yı > p2) A (p2 > p1), and (7)¢ is shorthand for —[7]-y. 

The formula [z]y is true in world w of M if for all v with (w, v) E |[7]|M 
it holds that ọ is true in v. This is completely axiomatised by the usual 
PDL rules and axioms (Segerberg, 1982; Kozen and Parikh, 1981): 


Modus ponens and axioms for propositional logic 
Modal generalisation From F y infer F [x]yp 


Normality F [z](e > 4) > (I[n]e > [7]¥) 
Test E [Poly > (p > h) 

Sequence F |r1; malp © [rilr]p 

Choice F [m Uma]p > ([ri]e A [ma]y) 
Mix E [n*]p => (eA [r|[x*]¢) 
Induction + (pA [a*](y > [z]~)) > [a*]e 


In (van Benthem et al., 2iiii6b) it is proved that extending the PDL language 
with an extra modality [A, e]y does not change the expressive power of the 
language. Interpretation of the new modality: [A, ely is true in w in M if 
success of the update of M with action model A to M ®A implies that y 
is true in (W,e) in M & A. To see what that means one has to grasp the 
definition of update models A and the update product operation ©, which 
we will now give for the epistemic preference case. 

An action model (for agent set J) is like an epistemic preference model 
for J, with the difference that the worlds are now called events, and that 
the valuation has been replaced by a precondition map pre that assigns to 
each event e a formula of the language called the precondition of e. From 
now on we call the epistemic preference models static modeis. 

Updating a static model M = (W, V,R, P) with an action model A = 
(E, pre, R, P) succeeds if the set 


{(w,e) Iw E W,e E E,M,w EF pre(e)} 
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abe oo wme )abc 


abc abc 


true ie abc 


FIGURE 1. Statie model and update. 


wme 


is non-empty. The result of this update is a new statie model M ®A 
(W! vi, RI, Pi) with 


e W'= {(w,e) Iw E W,e E E,M,w E pre(e)}, 


© VI(W,e) = V(w), 
e R; is given by {(w,e),(v,J)) I (w,v) E Ri, (e,J) E R4}, 
e PI is given by {(w, e), (v, J)) | (w, v) EPi, (e, J) E P;}. 


If the statie model has a set of distinguished states Wo and the action model 
a set of distinguished events Eo, then the distinguished worlds of M & A 
are the (W,e) with WE Wo and e E Eo. 

Figure 1 gives an example pair of a statie model with an update action. 
The distinguished worlds of the model and the distinguished event of the 
action model are shaded grey. Only the R relations are drawn, for three 
agents a, b,c. The result of the update is shown in Figure 2, on the left. 
This result can be reduced to the bisimilar model on the right in the same 
figure, with the bisimulation linking the distinguished worlds. The result of 
the update is that the distinguished "wine" world has disappeared, without 
any of a, b,c being aware of the change. 

In LCC, action update is extended with factual change, which is handled 
by propositional substitution. Here we will consider another extension, with 
preferenee change, handled by preferenee substitution (first proposed by 
van Benthem and Liu, 2004). A preferenee substitution is a map from 
agents to PDL program expressions zt represented by a finite set of bindings 


{ij = 17,...,in H Tm} 


where the // are agents, all different, and where the m; are program ex- 
pressions from our PDL language. It is assumed that each i that does 
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= 
abc 
ab abe 
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a 
abc a 
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\ wine j 
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water 
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abe abc 
ab: 
ne 


WE 


FIGURE 2. Update result, before and aftel' reduction under bisimulation. 


not occur in the lefthand side of a binding is mapped to >;. Call the set 
{i E Z | p(i) Æ >;} the domain of p. IfM = (W,V,R, P) is a preference 
model and p is a preference substitution, then MP is the result of changing 
the preference map P of M to P” given by: 


PP(i) Pi for i not in the domain of p, 
PP(i) feta)“ for i = ij in the domain of p. 


Now extend the PDL language with a modality |p] for preference change, 
with the following interpretation: 


M,w Elop :— MPwE ye. 


Then we get a complete logic for preference change: 


Theorem 3.1. The logic of epistemic preference PDL with preference change 
modalities is complete. 


Proo]. The preference change effects of |p] can be captured by a set of re- 
duetion axioms for lp] that commute with all sententiallanguage constructs, 
and that handle formulas of the form [pi|7|y by means of reduction axioms 
of the form 


lelle = [F,()llely, 
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AN AN 


FIGURE 3. Preorder with a non-transitive symmetrie closure. 


with F, given by: 


Fpi) ~i 
pä) if i in the domain of p, 
Fp(2s) fo; otherwise, 
Fp(?cp) [plep, 
Fp(m; 2) F (11); F p(t), 
F(T U m2) F(T) U Fm), 
For*) (Fol). 


It is easily checked that these reduction axioms are sound, and that for each 
formula of the extended language the axioms yield an equivalent formula 
in which IP] occurs with lower complexity, which means that the reduction 
axioms can be used to translate formulas of the extended language to PDL 
formulas. Completeness then fol!ows from the completeness of PDL. Q.E.D. 


4 Vet another logic... 


In this section we look at a more constrained case, by replacing the epis- 
temic preferenee models by 'belief revision modeis' in the style of Grove 
(1988), Board (2002), and Baltag and Smets (2006, 2008b) (who cal! them 
'multi-agent plausibility frames'). There is almost complete agreement that 
preferenee relations should be transitive and refiexive (preorders). But tran- 
sitivity plus refiexivity of a binary relation R do not together imply that 
Ru FR is an equivalence. Figure 3 gives a counterexample. The two extra 
conditions of weak connectedness for Rand for R` remedy this. A binary 
relation R is weakly connected (terminology of Goldblatt, 1987) if the fol- 
lowing holds: 


Yx, V, z((xRy \ xRz) — (yRzV y = zV ZRy)). 


Theorem 4.1. Assume R is a preorder. Then Ru R` is an equivalence iff 
both Rand č are weakly connected. 
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> 


FIGURE 4. Locally connected preorder that is not connected. 


Proo]. =>: immediate. 

<: Let R be a preorder such that both Rand R“ are weakly connected. 
We have to show that RUR” is an equivalence. Symmetry and refiexivity are 
immediate. For the check of transitivity, assume xR U y and yR U Rz. 
There are four cases. (i) xRyRz. Then xRz by transitivity of R, hence 
xRU tr« (ii) cRyRz. Then yR'x and yR`z, and by weak connectedness 
of R, either (cR°z or zR x), hence xR U R:z; or x = z hence xRz by 
refiexivity of R. Therefore xR U R°z in all cases. (iii) eR yRz. Similar. 
(iv) syk z. Then zRyRx, and zRx by transitivity of R. Therefore 
xRU RZ. Q.E.D. 


Call a preorder that is weakly connected and conversely weakly con- 
nected locally connected. The example in Figure 4 shows that locally con- 
nected preorders need not be connected. Taking the symmetrie closure of 
this example generates an equivalence with two equivalence classes. More 
generally, taking the symmetrie closure of a locally connected preorder cre- 
ates an equivalence that can play the role of a knowledge relation defined 
from the preferenee order. To interpret the preferenee order as conditional 
belief, it is convenient to assume that it is also well-founded: this makes for 
a smooth definition of the notion of a 'best possible world’. 

A belief revision model M (again, for a set of agents 1) is a tuple 
(W, V, P) where W is a non-empty set of worlds, V is a propositional valua- 
tion and P is a function that maps each agent i to a preferenee relation <; 
that is a locally connected well-preorder. That is, <; is a preorder (refiexive 
and transitive) that is well-founded (in terms of <i for the strict part of 
<;, this is the requirement that there is no infinite sequence of wz, w2, ... 
with ... <i w2 <i w1), and such that both <; and its converse are weakly 
connected. 
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In what follows we will use <i with the meaning explained above, >; for 
the converse of <;, >: for the converse of <i, and ~; for <; U >;. 

The locally connected well-preorders <; can be used to induce accessibil- 
ity relations =P for each subset P of the domain, by means of the following 
standard definition: 


> 


oP := {(æ,y) |£ viy Ay E MIN<,;P}, 


t 


where MIN <,P, the set of minimal elements of P under <;, is defined as 
{se P:Ys € P( <s> s< s')}. 


This picks out the minimal worlds linked to the current world, accord- 
ing to <;, within the set of worlds satisfying [v]M. The requirement of 
wellfoundedness ensures that MIN <P will be non-empty for non-empty P. 
Investigating these +” relations, we see that they have plausible properties 
for belief: 


Proposition 4.2. Let < be a locally connected well-preorder on S and let 
P be a non-empty subset of S. Then —>” is transitive, euclidean and serial. 


Proo]. Transitivity: if x =” y then y -x and y E MIN <P. Ify —P z then 
zr» y and z E MIN <Po It follows by local connectedness of < that z -+ x 
and by the definition of 3” that x =>” z. 

Euclideanness: let x >” y and x —? z. We have to show y =>” z From 
x oP y Y. x andy E MIN<P. From x >P Z Z.. x and z E MIN<P. 
From local connectedness, y ~ z. Hence y =>” z. - 

Seriality: Let x E P. Since < is a preorder there are y E P with 
y <x. The wellfoundedness of < guarantees that there are < minimal 


such y. Q.E.D. 


P 


Transitivity, euclideanness and seriality are the frame properties correspond- 
ing to positively and negatively introspective consistent belief (KD45 belief, 
Chellas, 1984). 

Figure 5 gives an example with both the < relation (shown as solid 
arrows in the direction of more preferred worlds, i.e., with an arrow from x 
to y for x > y) and the induced — relation on the whole domain (shown as 
dotted arrows). The above gives us in fact knowledge relations ~; together 
with for each knowledge cell a Lewis-style (1973) counterfactual relation: a 
connected well-preorder, which can be viewed as a set of nested spheres, with 
the minimal elements as the innermost sphere. Compare also the conditional 
models of Burgess (1981) and Veltman (1985) (linked to Dynamic Doxastic 
Logic in Girard, 2007). 

Baltag and Smets (2008b,a) present logics of individual multi-agent be- 
lief and knowledge for belief revision modeis, and define belief update for 
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FIGURE 5. Preference (solid arrows) and belief (dotted arrows). 


this as a particular kind of action update in the style of (Baltag et al., 1998), 
called action priority update. Here we sketch the extension to a system that 
also handles common knowledge and common conditional belief, and where 
the action update has belief change incorporated in it by means of relational 
substitution. 

The set-up will be less general than in the logic LCe: in LCC no assump- 
tions are made about the update actions, and so the accessibility relations 
could easily deteriorate, e.g., as a result of updating with a lie. Since in 
the present set-up we make assumptions about the accessibilities (to wit, 
that they are locally connected well-preorders), we have to ensure that our 
update actioris preserve thpsp relational properties. 

Consider the following slight modification of the PDL language (again 
assume p ranges over a set of basic propositions Prop and i over a set of 
agents 1): 


p Tip l> lyi A p2 lfr] 
tt ~i l< 1>; I? IA? IC |? | m1; T2 | m1 Uma lze" 


Call this language Lpref. This time, we treat ~; as a derived notion, by 
putting in an axiom that defines ~; as <; U>;. The intention is to let ~; be 
interpreted as the knowledge relation for agent i, <; as the preference rela- 
tion for i, >; as the converse preference relation for i, >’ as the conditional 
A as its converse, and 
Cas global accessibility. We use —; as shorthand for =. 

We have added agiobal modality C, and we will set up things in such way 
that |G]p expresses that everywhere in the model y holds, and that (G)yp 
expresses that p holds somewhere. It is well-known that adding agiobal 


modality and converses to PDL does not change its properties: the logic re- 


belief relation defined from <; as explained above, — 
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mains decidabie, and satisfiability remains EXPTIME-complete (Blackburn 
et al., 2001). 

The semantics of Lpref is given relative to belief revision models as indi- 
cated above. Formula meaning [y]|™ and relational meaning [[7]|™ are han- 
dled in the usual way. The interpretation of the knowledge relation of agent 
i is given by |~;]M := <M u >M, that for the preference relation of agent 
i by [<M = gM that for the converse preference relation of agent i by 
its converse, that for the conditional belief of agent i by [7] le" 
that for —f by its converse. The global modality is interpreted as the uni- 
versal relation, and test, sequential composition, choice and Kleene star are 
interpreted as usual. 

The interplay between the modalities [~;] (knowiedge) and [>,] (safe 
belief) is analysed by Baltag and Smets (2008b), where they remark that 
the converse preference modality [>,] in belief revision models behaves like 
an S4.3 modality (reftexive, transitive and not forward branching), and lives 
happily together with the S5 modality for |~;]. 

To see how this all works out, let us have a look at the truth conditions 
for [+?]y. This is true in a world w in model M if in all worlds v with 
v ~; wand v minimal in [[y]]M under <; it holds that 7 is true. This is 
indeed conditional belief, relative to y. Compare this with [>,]w. This is 
true in a world w if in all worlds that are at least as preferred, w is true. 
Finally, RAK is true in w if w is true in all worlds, preferred or not, that i 
can access from w. 

As a further example, consider a situation where Alexandru is drinking 
wine, while Jan does not know whether he is drinking wine or beer, and 
Sonja thinks that he is drinking tea. The actual situation is shaded grey, 
Jan's preference relation has solid lines, that of Sonja dotted lines. Reftexive 
arrows are not drawn, so Alexandru's preferences are not visible in the 
picture. 


3 


wme 


In the actual world it is true that Jan knows that Alexandru knows what 
Alexandru is drinking: |[~;]([~a]w V |~a]b), and that Sonja believes Alexan- 
dru is drinking tea and that Alexandru knows it: [s|[~,|é. Under condi- 
tion ,ź, however, Sonja has the belief in the actual world that Alexandru 
is drinking beer: [—,*]b. Moreover, Jan and Sonja have a common be- 
lief under condition , that Alexandru is drinking wine or beer: |—7* U 


35; (37' U>,")*|(w V b). As a final illustration, note that [+-8+ is true 
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in a world if this is noi among Sonja's most preferred worlds. Notice that 
if Sonja conditionalizes her belief to these worlds, she would believe that 
Alexandru is drinking beer: e ~|b is true in the actual world. 

It should be clear from the example that this language is very expressive. 
To get at a complete logic for it, we need axioms and rules for propositional 
logic, S5 axioms for the global modality (Goranko and Passy, 1992), axioms 
for forward connectedness of > and of < (see Goldblatt, 1987), axioms for 
converse, relating < to > and — to +, as in temporal logic (Prior, 1967), 
and the general axioms and rules for PDL (Segerberg, 1982). Finally, add 
the following definition of conditional belief in terms of knowledge and safe 
belief that can already be found in (Boutilier, 1994) as an axiom: 


(mie > (me A [Zille - 4). 


This definition (also used in Baltag and Smets, 2008b) states that condi- 
tional to y, i believes in w if either there are no accessible y worlds, or 
there is an accessible y world in which the belief in p — w is safe. The full 
calculus for Lpref is given in Figure 6. 


Theorem 4.3. The axiom system for Lpref is complete for belief revision 
modeis; Lpref has the finite model property and is decidabie. 


Proo]. Modify the canonieal model construction for modallogic for the case 
of PDL, by means of Fischer-Ladner closures (Fischer and Ladner, 1979) 
(also see Blackburn et al., 2001). This gives a finite canonieal model with 
the properties for <; and >; corresponding to the axioms (since the axioms 
for <; and >i are canonieal). In particular, each >i relation will be reftexive, 
transitive and weakly connected, each relation <; will be weakly connected, 
and the <; and >; relations will be converses of each other. Together this 
gives (Theorem 4.1) that the <; U >; are equivalences. Since the canonical 
model has a finite set of nodes, each <; relation is also well-founded. Thus, 
the canonical model is in fact a belief revision model. AIso, the —,; and 
+—; relations are converses of each other, and related to the >; relations 
in the correct way. The canonical model construction gives us for each 
consistent formula y a belief revision model satisfying y with a finite set 
of nodes. Only finitely many of the relations in that model are relevant 
to the satisfiability of y, so this gives a finite model (see Blackburn et al., 
2001, for further details). Since the logic has the finite model property it is 
decidabie. Q.E.D. 


Since the axiomatisation is complete, the S5 properties of ~; are deriv- 
able, as weil as the principle that knowledge implies safe belief: 


mile > [2:]¢- 
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Modus ponens 


Modal generalisation 


Normality 

Inclusion of everything in G 
Refiexivity of G 
Transitivity of G 

Symmetry of G 

Knowledge definition 
Truthfulness of safe belief 
Transitivity of safe belief 

> included in <“ 

< included in >` 

Weak connectedness of < 
Weak connectedness of > 
Conditional belief definition 
— included in —“ 

+ included in >” 

Test 

Sequence 

Choice 

Mix 


Induction 
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and axioms for propositional logic 


From F y infer F [z]y 


F [ri > 4) > (tle > [x}y) 
Gly > [rly 

Gle > ẹ 

F- [G] — [C]C]ep 

F p> [C] (Gp 

F [mile > [Si U Daly 

F [>de > ge 

F [zde > [lede 


by [Ze 

F g > [<e 

[Sille A ide) > y) Vi A dy) > 9) 
[zdle A Bie) > h) v l A dy) > 9) 
E [>f] = ((~ ae —> ve) [>] > ))) 
Salata Pp 

Fes [HPP ey 

F [Pele > (y > 4) 


F [mm ; male © [m] 

F [ry Urey  ([m] eA [ra]y) 
F [n*y > (pA [a|[n*]¢) 

F (p Alt] > file) > [n*]¢ 


FIGURE 6. Axiom system for Lpref. 
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The same holds for the following principles for conditional belief given in 
(Board, 2002): 


7l 


Safe belief implies belief F [>;]y > [> ]y 

Positive introspection H 7] > [~il >] 

Negative introspection } -[3?]ye = Imi] 

Successful revision F [>f]e 

Minimality of revision H (=f) = (j2^?]x = [>f] — X)) 


We end with an open question: is >f definable from >; and <; using only 
test, sequence, choice and Kleene star? 


5 Combining update and upgrade 


The way we composed knowledge and belief by means of regular operations 
may have adynamie fiavour, but appearance is deceptive. The resulting 
doxastic and epistemic 'programs' still describe what goes on in a statie 
model. Real communicative action is changing old belief revision models 
into new ones. These actions should represent new hard information that 
cannot be undone, but also soft information like belief changes that can 
be reversed again later on. For this we can use update action by means 
of action modeis, with soft information update handled by means of action 
priority update (Baltag and Smets, 2008b,a), or preferenee substitution as 
in (van Benthem and Liu, 2004). Here we will propose a combination of 
these two. 

Action models for belief revision are like belief revision modeis, but 
with the valnation replaced by a precondition map. Wp add two extra 
ingredients. First, we add to each event a propositional substitution, to be 
used, as in LCC, for making factual changes to statie modeis. Propositional 
substitutions are maps represented as sets of bindings 


{Pl} Y1,- -Pn = Yn} 


where all the Pi are different. Itis assumed that each P that does not occur in 
the lefthand side of a binding is mapped to p. The domain of a propositional 
substitution øg is the set {p E Prop | o(p) Æ p}. IfM = (W, V,F) is a belief 
revision model and øg is an [Pref propositional substitution, then Vý is the 
valuation given by AwàÀp : wE |p" ]M. In other words, Vg; assigns to w the 
set of basic propositions p such that p” is true in world w in model M. M7 
is the model with its valuation changed by o as indicated. Next, we add 
relational substitutions, as defined in Section 3, one to each event. Thus, 
an action model for belief revision is a tuple A = (E,pre,P,psub,rsub) 
with E a non-empty finite set of events, psub and rsub maps from E to 
propositional substitutions and relational substitutions, respectively, and 
with rsub subject to the following constraint: 
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1:p 


FIGURE 7. Unconstrained relational substitution creates havoc. 


If e ~; f in the action model, then rsub(e) and rsub(f) have the 
same binding for i. 


This ensures a coherent definition of the effect of relational substitution on 
a belief structure. The example in Figure 7 il!ustrates this. But note that 
the substitutions are subject to further constraints. In the action model in 
the example, a single agent has a preference for -'p over p. In the update 
model, a substitution reverses the agent's preferences, but the agent cannot 
distinguish this from an action where nothing happens. What should the 
result of the update look like7 E.g., is there a preference arrow from (2,3) 
to (1,4)7 This is impossible to answer, as action 3 asks us to reverse the 
preference and action 4 demands that we keep the initial preference. The 
constraint on substitutions rules out such dilemmas. 

The relational substitution p = rsub(e) at event e in action model A is 
meant to to be interpreted 'Iocal!y' at each world w in input model M. If 
P is the preference map of M, then let PẸ be given by: 


P(t) Pin [whim for i not in the domain of p, 
Po) LOIT n beling 
for i = ij in the domain of p. 


Thus, Pf is the result of making a change only to the local knowledge cel! 
at world w of agent i (which is given by the equivalence class |w|j.,)m). 


Let 
U ec) 
wew 
Then P" (i) gives the result of the substitution pon P(1), for each knowledge 


cel! |w|y..,jm for i, and P” gives the result of the substitution pon P, for 
each agent i. 
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Now the result of updating belief revision model M = (W, V, P) with ac- 
tion model A = (E, pre, P, psub, rsub) is given by M @A = (w! VI, Pi), 
where 


- Wl- {(w,e) we Wee E,M,wk pre(e)}, 
* VI(W,e) = V” (w), 


e PI(i) is given by the anti-lexicographical order defined from PP(i) and 
P(i) (see Baltag and Smets, 2008b,a). 


With these definitions in place, what are reasonable substitutions? A pos- 
sible general form for a preference change could be a binding like this: 


>; > [pi 92,---, Pnl. 


This is to be interpreted as an instruction to replace the belief preferences 
of i in the local knowledge cells by the new preference relation that prefers 
the yı states above everything else, the ~y; A ya above the ~y; A 7p 
states, and so on, and the 7y; A agyz A... A mYPn—1 A Yn states above the 


ap; A agp A... A ayy states. Such relations are indeed connected well- 
preorders. 
most LEE EE EEI LEESE EEI least 
preferred preferred 
Note that we can take [y), %2, ---, Pn] as an abbreviation for the following 


doxastic program: 


CR) 


(~i; ?p1) U 
U (22913? p2; ~i; ? mp1; 2mp; 2H3) 
U 
U 


(7G 13+ ++ Pn; ~i; y1; e Pn -1; Pn) 


In general we have to be careful (as is also observed in (van Benthem and 
Liu, 2004)). If we have a connected well-preorder then adding arrows to 
it in the same knowledge cell may spoil its properties. Also, the union of 
two connected well-preorders need not be connected. So here is a question: 
what is the maximal sublanguage of doxastic programs that still guarantees 
that the defined relations are suitable preference relations? Or should belief 
revision models be further constrained to guarantee felicitous preference 
change? And if so: how? 
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6 Examples 


Global amnesia: the event of agent a (Jason Bourne) forgetting all his 
beliefs, with everyone (including himself) being aware of this, is represented 
by the following action model (for the case of three agents a, b,c): 


a:=G )abc 


Alzheimer: the event of agent a forgetting everything, with the others 
being aware of this, while a wrongly believes that nothing has happened. It 
is tempting to model this with the following update model: 


abc abc 


Note however that this does not satisfy the constraint on relation update 
(the two actions are connected, but the substitution for a is not the same), 
so it may result in incoherent modeis. 

Lacular amnesia (specific forgetting): forgetting everything about p. 
One way to model this is by means of an action model with a single action, 
accessible to all, with the relational substitution 


2, bo (>, U eogi apy” 


This will effectively add best-world arrows from everywhere in the knowl- 
edge cell to all ~p worlds. 


Confession of faith in p, or publicly accepting p: an action model 
with a single action, accessible to all, with the relational substitution 


Di (24 U (2p; ~i; 2p)". 


This will make the p worlds better than the ~p worlds everywhere. 
Submission to a guru: the act of adopting the belief of someone else, 
visible to all. A problem here is that the guru may know more than I do, so 
that the guru's preferences within my knowledge cell may not be connected. 
This means that the substitution <i œ> <,;—the binding that expresses 
that i takes over j’s beliefs-may involve growth or loss of knowledge for i. 
Consider the example of the wine-drinking Alexandru again: if Jan were to 
take over Sonja's beliefs, he would lose the information that Alexandru is 
drinking an alcoholic beverage. 
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Conformism: adopting the common beliefs of a certain group, visible to 
all: an action model with a single action accessible to all, with the following 
substitution for conformist agent i: 


>; te (>; U >); (2: U >j)". 


Belief coarsening: the most preferred worlds remain the most preferred, 
the next preferred remain the next preferred, and all further distinctions 
are erased. An action model with a single action accessible to all, and the 
following substitution for agent i: 


>; e =>; U?T. 


The union with the relation ?T has the effect of adding all refiexive arrows, 
to ensure that the result is refiexive again. 

Van Benthem's fẹ is handled by a substitution consisting of bindings 
like this: 


2; = (Ppiri?ry) U (29; 2429) U (? >; 24? 79). 


This is an alternative for an update with an action model that has ~y <B y. 
The example shows that conservative upgrade is handled equally weil by 
action priority updating and by belief change via substitution. But belief 
change by substitution seems more appropriate for 'elite change’. For this 
we need a test for being in the best y world that i can conceive, by means 
of the Panglossian formula (-7)T. The negation of this allows us to define 
eliLe change like this: 


>; = PU (243 ?[-P]L). 


This promotes the best y worlds past all other worlds, while leaving the 
rest of the ordering unchanged. Admittedly, such an operation could also 
be performed using action priority updating, but it would be much more 
cumbersome. 


7 Further connections 


To conneet up to the work of Bonanno again, what about time? Note 
that perceiving the ticking of a clock can be viewed as information update. 
A clock tick constitutes a change in the world, and agents can be aware 
or unaware of the change. This can be modelled within the framework 


introduced above. Let ri, ...,f. be the clock bits for counting ticks in 
binary, and let C := C +1 be shorthand for the propositional substitution 
that is needed to increment the binary number f;,...,f, by 1. Then public 


awareness of the clock tick is modelled by: 
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c:-c-t abe 


Thus, perception of the ticking of a clock can be modelled as 'being in 
tune with change in the world’. Still, this is not quite the same as the ‘next 
instance’ operator O, for the DEL framework is specific about what happens 
during the clock tick, while 0 existentially quantifies over the change that 
takes place, rather in the spirit of (Baibiani et al., 2007). 

In belief revision there is the AGM tradition, and its rational reconstruc- 
tion in dynamic doxastic logic à la Segerberg. Now there also is a modal 
version in Bonanno style using temporallogic. It is shown in (van Benthem 
and Pacuit, 2006) that temporal logic has greater expressive power than 
DEL, which could be put to use in a temporal logic of belief revision (al- 
though Bonanno's present version does not seem to harness this power). As 
an independent development there is dynamic epistemic logic in the Amster- 
damjOxford tradition, which was inspired by the logic of public announce- 
ment, and by the epistemic turn in game theory, à la Aumann. Next to this, 
and not quite integrated with it, there is an abundance of dynamic logics 
for belief change based on preference relations (Spohn, Shoham, Lewis), and 
again the Amsterdam and Oxford traditions. I hope this contribution has 
made clear that an elegant fusion of dynamic epistemic logic and dynamic 
logics for belief change is possible, and that this fusion allows to analyze 
AGM style belief revision in a multi-agent setting, and integrated within a 
powerful logic of communication and change. 
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Abstract 


Recent developments in the interface of Economics and Logic yield 
the promise of capturing phenomena in strategie interaction that was 
previously beyond theoretical economie modeling. We consider one 
such application in the case of strategie situations that players may 
perceive differently. We show that the content of messages sent in 
this setting carries strategie implications without resorting to prior 
conventions and beyond the observations made in the literature deal- 
ing with cheap talk communications. The content of the message 
becomes strategically meaningful since it reveals perceptions. Vari- 
ous forms of interaction between meaningful statements and strategie 
behavior are discussed. 


1 Introduction 


The modeling of information in strategie settings has been the centerpieee 
of economie theory for almost four decades now. Models of signaling, bar- 
gaining, auctions, contracts, mechanism design and more, are grounded in 
the premise that the decision makers posses private information relevant for 
outcomes, that their behavior is conditioned on their information, that they 
form beliefs about the information held by others and that they revise these 
beliefs based on observed actions. In addition, these fields of research add 
the possibility of communication between decision makers. However, formal 
theoretical economics modeling tends to overlook the content, or meaning 
of the messages that the decision makers may exchange, and, in particular, 
how this meaning may inftuence strategie behavior. 

In his seminal work, Spenee (1973) introduced the job market signaling 
model. In this model an employer wishes to hire qualified employees, how- 
ever only the potential candidates know their private degree of qualification. 
Spenee shows that if the employees are able to signal their abilities then an 
equilibrium of the game may be able to separate the high quality candi- 
dates from the lower quality ones. This signal is a potentially costly action 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 105-119. 
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taken by the candidates, for example obtaining a college degree. The im- 
portant feature for separation by a signaling action is that it is more costly 
to a candidate that is less qualified for the job. The signaling action itself 
may have no impact on the employee's underlined qualities as they pertain 
to the job offered, it only serves as a way for the more qualified workers 
to separate themselves from the others. The information-the employee's 
qualification-is being deduced indirectly from his action in equilibrium. 
Such inferred information from an observed action is what economists usu- 
ally refer to as informational content.! The content is the indirect deduction 
about private information derived from the information dependent strategie 
incentives associated with various actions. 

This "(costly) action speaks louder than words" approach resonates weil 
with the economie perspective that a message that can be sent without 
incurring costs,2 has to be stripped of all meaning other than its strategie 
intention when analyzed by the receiver. With that in mind, the ingenious 
work by Crawford and Sobel (1982) demonstrated that cheap talk, as it 
became to be known, can have profound strategie impact. By reacting 
to different messages differently, the receiver can create an incentive for 
the sender to send different messages depending on the sender's private 
information which also makes the receiver's choices best responses and all 
this without incurring any cost of signaling. The strategie reaction in such 
cheap talk games distinguishes between various messages, but it need not, 
and does not, depend in any way on the content of a message-there is no 
relevanee to what is being said only to saying one arbitrary thing rather 
than the other. 

While cheap talk inlluences strategie outcornes, iL was recognized in Lhe 
refinement literature" that the content of the messages plays no role in the 
solution (see Cho and Kreps, 1987, for an illuminating explanation). One 
of the few direct efforts to confront the issue of meaningful messages in 
Economics is due to Farrell (1993). He suggests a delicate solution con- 
cept termed Neoloqgism-proof equilibrium where an equilibrium defines the 
expected messages (that must agree with equilibrium behavior of the var- 
ious sender types) and has to be robust to messages that are new-that 
will surprise the receiver given the expected equilibrium play. In this sense 
the language is endogenously dictated by some equilibrium behavior that 
forces the meaning of the message to coincide with that behavior. The work 
of Matthews et al. (1991) confronts the problem by introducing armounce- 


1 See Spence's (2001) Nobel Prize lecture for an insightful description of this type of 
informational content. 

2 In particular no indirect casts in the form ofa contractual or reputational commitment. 

3 The literature dealing with providing alternative solution concepts to Nash equilibrium 
that will confront the multiplicity of equilibria and remove seemingly "unwanted" 
behavior. 
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ment proof equilibria as weil as additional weak and strong versions. In 
these solutions the sender can make more delicate announcements of de- 
viation from the given equilibrium and they ask for behavior with various 
levels of robustness against the deviating announcements when it is assumed 
that deviations are interpreted correctly by the receiver. Inthe background 
there is a notion of convention that not only assumes asender will do what 
he says as long as there is no incentive to deviate, but that the receiver 
interprets the messages in the marmer intended by the sender. 

On the other hand, philosophers and, in particular, logicians studied 
both language, communication and interpretation long before it entered 
economie discourse. Moreover, there have been studies relating these to 
actions and mounting interest in the incorporation of strategie reasoning, 
as weil as the use of logic games (see van Benthem, 2007) which in turn 
is advancing game theoretical results. In relation to pre-play communi- 
cation in games the work of Lewis (1969) sets the stage. With Gricean 
maxims (cf. Grice, 1957) shedding light on methodical principles for for- 
mulating foundations for refinement conditions as described above. As in 
similar cases of intersecting disciplines, the interaction works both ways 
with game theoretical principles utilized on the linguistic side of communi- 
cation as seen in (Parikh, 2001), (van Rooij, 2003, 2004) and (Jager, 2006), 
with dynamics-evolutionary and learning-e-entering the scene and bridging 
semantics and pragmatics. 

Here we take a different approach to the strategie impact of content. 
We consider communication that may alter how decision makers perceive 
the strategie situation, or how they view its perception by others. It is the 
game Iorrn that rnay change as a result of cornrnunication. Using a recenLly 
developed framework for games with unawareness in (Feinberg, 2004, 2005b) 
we consider games where players may initially model the game differently, or 
view others as modeling it differently. In this paper we consider the impact 
of adding a communication stage to such games. The players can now reveal 
their perception. It turns out that some statements are inherently credible, 
without resorting to prior conventions or to credibility derived from the 
definition of the solution. As such, we find that the content of the message 
can inftuence the strategie behavior of the decision makers without resorting 
to a refinement or an arbitrary assignment of interpretations to statements. 

Our aim is to link the modeling of strategie communication in economics 
with an almost pragmatic interpretation of what constitutes a game which 
allows us to insert meaning to messages. We find this to be a continu- 
ation of previous explorations along the interface between economics and 
language" with emphasis on applicability to economie settings. The next 


4 We see the model here as putting pragmatism mueh closer to mainstream eeonomies 
when initially Rubinstein saw it as "the topie furthest from the traditional economie 
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section provides a game theoretical analysis of some games with unaware- 
ness to which we add a pre-play communication stage. We then discuss 
the aspects of strategie choice of messages in aspecific example of cal’ sales 
where a dialogue sets the bargaining game. 


2 Meaningful message 


Our starting point is a game with unawareness-a game where the players 
may have a partial description of the strategie situation, or where they 
attribute such limited perception to others, view others as perceiving them 
to be restricted and so on. As a leading example consider the strategie 
situation modeled in the game depicted in Figure 1 and Figure 2 below 
taken from (Feinberg, 2üü5a). The game in Figure 1 represents all the 
actions available to the two players and the payoffs associated with each 
action profile. Assume that Alice and Bob are both aware of all the actions 
available in the game. However, Alice's perception is that Bob is not taking 
into account all her actions. In particular she is confident that Bob models 
the game as depicted in Figure 2. Alice's perception can be due to her 
observing that action a3 was never taken in similar situations in the past, 
which would be supported by the observation that a3 is not part of a Nash 
equilibrium in the standard normal form game in Figure 1, or she may 
assume that Bob is relatively a novice in this strategie situation, or that this 
is her "secret" action, whichever the reason, Alice views Bob as unaware of 
this action. Hence, Alice's view is that Bob's view of the game includes only 
the actions {al,a2,b],b2,b3} and that the strategie situation he models is 
the game depicted in Figure 2. We further assume, as in (Feinberg, 2üü5a), 
that Bob actually views the game as Figure 1. He correctly recognizes that 
Alice considers all the actions. Furthermore, we assume that he recognizes 
that Alice is unaware that he is considering a3, i.e., Bob views Alice's view 
of his view of the game to coincide with Figure 2. Similarly, all other higher 
order iterations correspond to the game in Figure 2-Alice's view of Bob's 
view of Alice's view, Bob's view of Alice's view of Bob's view, Alice's and 
Bob's view of these views, and so on. 

The game depicted in Figure 1 has a unique Nash equilibrium (a2, b) 
obtained by iteratively eliminating strictly dominated strategies, hence a 
unique rationalizable outcorne". However, since Alice perceives Bob's view 
of the situation to be as depicted in Figure 2 she may believe that Bob is 
considering playing according to the Nash equilibrium (al, b2) of the game 
in Figure 2-the pareto optimal equilibrium. 


issues" in his book (Rubinstein, 2000). 
5 As noted above this may justify Alice's perception that a3 may be overlooked by 
others. 
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b | & | b 
al 0,2 3,3 0,2 
a2 2,2 2. 1 2. 1 
a3 | 1,0 | 4,0 | 0,1 


Alice 


FIGURE |1. 


Alice al | 0,2 | 3,3 | 0,2 
a2 | 22| 2,1 | 2,1 


FIGURE 2. 


Alice will be inclined to choose her best response to b) which is a3- 
her "secret" action. Bob can make the exact same deduction as we, the 
modelers, just did, since he is aware of all the actions and correctly perceives 
that Alice is unaware that he is considering that she may take the action a3. 
Hence, Bob can deduce that Alice assumes he plays b and she chooses a3. 
Bob's best response is to play bz. The outcome will be a best response based 
on a Nash equilibrium of the restricted game where Alice plays a3 and Bob 
plays bz. The decision makers' reasoning (incorporating Nash equilibria 
reasoning) may lead them to actions that are not part of an equilibrium 
neither in the game Figure 1 nor in Figure 2. We end up with the worst 
possible payoff for Alice and a low payoff for Bob, although both are aware 
of all possible actions in the game, both are commonly aware of the action 
profile of the unique Nash equilibrium (a2, b,) and both act rationally given 
their perceived view of the game. 

We wish to add pre-play communication to this game and consider the 
impact of messages on the strategie behavior. More precisely, assume that 
Bob can send a message to Alice. If Alice does not consider the possibility 
that Bob is actually aware of a3 there is no impact of sending messages 
that have no meaning as in cheap-talk communications since there are no 
differing types that Alice is considering and for which updated beliefs could 
result from messages sent by Bob. However, Bob can send the message "do 
not use action a3” or "I know about action a3” or even "don't think that 
l am not taking a3 into account". The specific reference to this action's 


name® requires Alice to reconsider her view of Bob's view of the game. The 


6 The reader ean replaee the narnes of aetions with an appropriate deseription for a 
specific strategie interaetion. It is here that Grieean pragmaties are invoked, see (Griee, 
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interpretation is that Bob has just related the action a3 to the strategie 
interaction. She has modeled the situation with Bob being unable, or, 
practically unable, or even unwilling, to reason about her "secret" action, 
and here Bob is explicitly mentioning this action in the context of this game. 
Can Alice ignore this statement? We believe the answer is negative. What 
forces Alice to revise her view is not the fact that a message was sent, nor is 
it some preconceived interpretation of the statement, it is that the content 
of the message indicates that Bob can reason about aspecific action a3 that 
she initially deemed relevant to the specific strategie interaction. 

A formal framework for Alice's revision due to such a message follows 
from the representation of a game as a formal language as in (Feinberg, 
2005b) when extended to restricted languages as in (Feinberg, 2004). Hence, 
the interpretation of a message can be a demonstration of the cognitive abil- 
ity of the sender, in the sense of the extent of his ability (or his modeling 
choice) when reasoning about the situation at hand. With this interpre- 
tation, Bob can send a statement that includes the term a3 only if a3 is 
part of the language he employs in reasoning about the strategie situation 
at hand. Hence, Alice who already possesses a3 in her description of the 
situation, must deduce that a3 is part of Bob's language for the game which 
is a revision of her initial perception. 

What follows aftel' Bob mentions a3 and Alice revises her perception of 
Bob is the play of the unique Nash equilibrium of the game in Figure 1. In 
this specific example, Alice is motivated to choose a2 even if she believes that 
Bob would not believe that she will revise her perception of his perception, 
i.e., even if Bob would still choose b; expecting her to choose a3 she now 
realizes that this is possible, making a2 a best response. WhaLever Lhe 
update of perception is, our main claim is that the content of a message can 
change the scope of reasoning that decision makers attribute to each other 
and hence dramatically inftuence strategie behavior and outcomes. 

In this example we assumed that a3 is not some generic statement. When 
Bob mentions a3 Alice must find it is highly unlikely that he was able to 
guess it. 1VJIoreover, even if Alice suspects that Bob is lucky she would realize 
that he might consider such actions possibie, leading her to the (a2, by) 
solution once more. Our observation relies on the message containing a3 
and not on any promised action, or intention Bob expresses. If anything, 
when Alice hears Bob warning her not to take action a3 she might even 
deduce something about Bob's reasoning about high order perceptions in 
this situation. She might realize that Bob is able to deduce that she is 
contemplating a3 hence he might actually realize that she initially did not 
realize he is reasoning about a3. Even without any warning attached, Bob's 
incentive for mentioning a3 may lead Alice to deduce that Bob is not only 


1957) and in the context of game see (van Rooij, 2003). 
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aware of a3 but that he realizes that she perceived him to be unaware of it. 
Since, otherwise, why would he choose to mention a3 of all things? While 
this transmission of higher order perception will not impact the outcome in 
this case, assuming Alice realizes that Bob considers a3 to begin with, it 
could be of strategie impact in other settings. It also sheds some light on 
the various notions of credibility associated with a message and the derived 
high order perceptions. 

There are two different notions of credibility that can be attributed to 
a statement made by Bob. The first is the one that Farrell (1993) (see 
also Myerson, 1989) suggested which corresponds to a weil developed body 
of work in logic-see (Grice, 1957), (Lewis, 1969) and more recent results 
in (van Rooij, 2003). It corresponds to the case where Bob makes the 
statement "I am going to play bs". Alice is assumed to interpret it according 
to the convention that Bob is indeed intending to play the action with 
the same name and credibility is generated when it survives the economie 
incentive criteria for Bob once Alice acts rationally and assumes that he 
speaks truthfully-robustness to deviation. The second notion of credibility 
of a statement we introduced here corresponds to a statement whose content 
reveals perception, or ability, that may not have been assumed to begin with. 
When Bob mentions action a3 he expressed awareness that was beyond 
Alice's perception of Bob. This may lead her to assume that he is modeling 
the game with action a3 and the exact marmer in which a3 is stated, e.g., 
as a promise or a threat, matters less. To distinguish from the first notion 
of credibility we call a statement meaningful if it conveys an aspect of the 
perception of a strategie situation. 

InLermediaLe notions between credible and meaningful talk also arise. 
For example, a player may want to describe their own action when they 
perceive that others do not incorporate it into their view of the game. But 
here, there must be some evidence convincing the other side. Such evidence 
maybe exogenous (e.g., demonstrating a proof of the existence of the action), 
as in a convention, or be supported from intrinsic strategie motivation. 
In the example above, when Bob mentioned a3 the statement was more 
meaningful due to its credibility, i.e., Alice had a good reason to believe 
that Bob was not just guessing, because strategically it is beneficial for Bob 
to make such a statement if he is aware of a3, while if he thinks the game 
is as in Figure 2 then even if he convinces Alice to believe in a fictitious 
a3 he would, from the limited point of view, lose by leading to the lower 
payoff equilibrium in Figure 2. Alice should correctly retrieve its meaning- 
update her view of Bob's perception of the game---since this deduction is 
also supported when considering Bob's incentives. 

In general, the interaction of credibility and meaning may be more in- 
tricate and sometimes conftieting. For example, consider the game depicted 
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Bob 
b | b2 | b 
Alice al | 0,0 | 3,5} 1,1 
a2 | 5,2 | 2,1 | Ox 


FIGURE 3. 


in Figure 3. Assume that in this game Alice is unaware of b and perceives 
the game as in Figure 4 where two pure strategy Nash equilibria compete 
for coordination. The strategy profile (a2,b,) is much more attractive to 
Alice in Figure 4. Assume that 5 > x > 2 then if Bob says "do not play 
a2 because I will never play b: since I have a strategy b3 that dominates 
it" (or simply says "I have b") then Alice may consider a revision of the 
perception of the game. The description of b mayor may not convince 
her. But the situation becomes puzzling if x > 5. In this case Bob has no 
incentive telling Alice about this action-he would rather have her choose 
a2. The problem is what should Alice deduce if he does describe action bz. 
Should she revise her perception, or should she just assume that he is trying 
to lead her to the solution of Figure 4 which benefits him? How should she 
weigh any evidence of the existence of b; that Bob may produce? Her prob- 
lem is that if she accepts his statement as meaningful it would be irrational 
for him to make it, if she doesn't, then stating it is no longer irrational. 
Hence, the level of verifiability of the content may mix with the notion of 
credibility of statements and with the meaningfulness of a statement which 
in turn determines its credibility. 

To complicate things even further, we point out that credibility and 
meaning can interact with the incentives for making any statement at all. 
If one assumes credibility to begin with’ it might inftuence deduction about 
the perception of others. Bob would rather not reveal 6; when x > 5, but 
the strategic situation might force him to make a statement about actions. 
If he believes that Alice will interpret his statements as truthful unless they 
are not credible, then he has to choose between saying "I will play b," and 
"I will play bb". If he says the second then Alice will find it credible since 
she views the game as Figure 4, but Bob would lose the opportunity of 
gaining more than 5. However, if he says he will play b: she would find it to 
be non-credible since having the opportunity to determine the focal point 
according to the convention, Bob should be strictly better off declaring bp 
in Figure 4. In this case Bob would like to have the right to be silent, 
but only if Alice does not realize that Bob had the opportunity to make a 


7 In the traditional sense that a stated action is played if there is an incentive to follow 
through if it is believed. 
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Bob 
b bə 
Alice a/ | 0,0 | 3,5 
a2 | 5,2 | 2,1 
FIGURE 4. 


statement. In this case, the perception of uilietlier statements can be made 
may inftuence the strategie behavior. 

This example leads to an additional twist which concerns with how cred- 
ibility may trigger reasoning. If Alice reasons by assuming credibility then 
when Bob armounces "I will play bj" he might trigger her to wonder if she 
is missing something in the strategie environment. This in turn may lead 
her to discover the missing action or develop beliefs about the consequences 
of its existence. She might very weil conclude that she is better off playing 
a/ (maybe even declaring it). But if this is the case, we may ask whether 
we could have a game with no unawareness at all where Bob would delib- 
erately make an announcement that may cause Alice to believe that she is 
missing some aspect of the game, leading her to behavior that Bob could 
not induce otherwise. The conceptual answer seems negative since if the 
armouncement eventually leads Alice to behave in a way that benefits Bob 
beyond any believed announcement, then it is actually a credible declara- 
tion by Bob and Alice's reasoning should unravel it. However, we lack the 
formal machinery for discovery in this setting. 

We note that in a dynamic setting meaningful indirect deductions can be 
made not only from statements but also from observing actions. Consider 
the first example where Alice assumes that Bob views the game as Figure 2. 
Assume the game is repeated twice but that only Alice can observe the 
choices made in the first round before playing the second round, i.e. even if 
Alice chooses a3 in the first round, Bob will not become aware of a3 in the 
second round if he was previously unaware of it. In this case, even without 
communication, if Alice observes Bob choosing b; in the first round she can 
either assume that Bob is irrational since b is dominated in Figure 2, or she 
can deduce that Bob is aware of a3 and followed the reasoning described in 
the beginning of this section. 

We reiterate that the use of a variety of messages types in these examples 
such as "do not choose a3", "I know about a3", "I realize you are considering 
a3" and "I am going to do... because of a3" all have the same impact in 
the sense of meaningful talk. It is the mentioning of a3 in any form which 
conveys an ability to reason about it in the specific context and impacts 
the perception of the game leading to strategie deductions. The messages 
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become reliable indications of the extent of the language that the decision 
maker employs in the situation. In general, both messages and actions can 
inftuence perceptions, deductions about others perceptions and so on. The 
phrases in the language which describe the situation from an economie view 
point-the game that is being played-convey meaning because they reveal 
the perception of the game which may not be commonly known. 


3 Negotiating a bargain 

The interaction of meaningful talk and strategie behavior is not limited to 
pre-play communication. As indicated above the strategie choice of state- 
ments takes into account the derived meaning by the receiver. Furthermore, 
since meaningful statements change the perception of the strategie situation, 
they could be highly sensitive to the choices made in adynamie setting-a 
dialogue. 

To illustrate some of the strategie features of a dialogue consider the 
case of a new cal' purchase. Alice is an inexperienced buyer who is facing 
the experienced cal’ salesperson Bob. Before explicit bargaining over the 
price begins (or while price discussions are held), there is a dialogue in 
which statements are made, questions are asked and answers are given. 
The strategie decision making involved in this process is far from trivia!. 
We focus only on a partial number of dimensions of the cal’ purchasing case 
with emphasis on the items that set the beliefs over reservation prices, most 
notably the dealer's reservation prices. Throughout this story we assume 
that Bob is fully aware of all possible dimensions of the interaction, he will 
not be informed of Alice's private information and may possess uncertainty 
as to Alice's perception of the situation, but he has full command of what 
Alice might be reasoning about. 

We simplify the situation further by assuming that Alice made her choice 
of vehicle she wishes to purchase", the cal’ is available at the dealership and 
Alice has no trade-in. Alice may be unaware of the following dimensions of 
the transaction: 


1. The sticker price on the cal’ (the MSRP) has no immediate relation 
to the dealer's cost or relationship with the manufacturer. 


2. There is an "invoice price" which is the basis for the cost of the cal’ 
to the dealer. 


3. There are adjustments to the dealer's invoice, such as quota incentives, 
year end bonuses, opportunity costs (new modeis, lot space). 


8 This is astrong assumption as in many cases salespeople have an extensive range of 
inftuence on how the buyer perceives and compares potential substitutes. 
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4. The actual cost of various additional options (such as fabricjpaint 
protection treatments, extended warranties). 


5. The existence of a nearby competing dealership selling an identical 
cal’. 


6. The possibility of purchasing the cal’ over the Internet. 


7. The possibility of financing a cal’ purchase in favorable terms com- 
pared to unsecured consumer credit such as credit card purchases. 


8. The information the salesperson has regarding the dealership's costs. 


We use this partial list of concrete attributes of the specific strategie sit- 
uation to highlight some aspects of a strategie dialogue that incorporates 
meaningful content being revealed via statements. 

Being the more experienced party, the salesperson's problem is to dis- 
cover not only as much as possible about Alice's reservation prices (assuming 
her taste was completely revealed by the choice of cal' and model), but also 
to inftuence her perception of the strategie situation in a marmer that puts 
her at a disadvantage in the bargaining stage and eventually leads to the 
highest profits to the dealership". This may amount to things as obvious 
as trying to sell options as in 4 aftel' the price has been agreed upon, but it 
also relates to inftuencing how Alice models the bargaining game itself, in 
particular, what bounds she sets on the dealership costs. Alice, on the other 
hand, must realize she is in a situation where there are potential aspects 
she might be missing, aftel' all she does not buy cars very often and should 
assume that Bob is experienced.1" 

We list various aspects which are relevant to a strategie dialogue with- 
out formulating a detailed decision tree, in which the choice of statements 
and questions are presented and the two parties perception of the game 
(including higher order perceptions) evolves according to these statements. 
Instead, we focus on various considerations that inftuence the optimal choice 
of statements, this allows us to capture a variety of scenarios without de- 
tailing the full scope of the game for each one. We refer to (Feinberg, 2004) 
for an illustration of how these dynamic games with unawareness can be 
constructed. 

The first aspect is the timing of bargaining and the importance of having 
the dialogue at all. As we will see, Bob can strictly benefit from obtaining 
information from Alice and hence has an incentive to engage in a dialogue 


9 The salespersen commission is increasing in the dealership profit from the sale. 

10 This is a very particular notion of experience, as it relates to the magnitude of the set 
of attributes of the situation that a decision maker is aware of, as weil as the ability 
to consider the limited perception of the buyers. 
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before discussion of prices occurs. For example, if Alice asks what the price 
is, Bob would refer her to the sticker price, or may even delay the answer 
by stating it depends on other factors, such as financing, options etc. We 
conclude that the ability to lead the other party into the dialogue has value 
on its own. In particular, there will be use of statements that might try and 
convince Alice as to why a lengthier discussion is required. 

Much of the information on Alice's perception can be gained by Bob 
from indirect questions. Indirect questions are preferabie for Bob since they 
are less likely to reveal the very same aspect of which Alice may be unaware. 
For example, to adjust his beliefs on whether Alice is aware of 5 Bob may 
prefer to ask "have you driven this cal' before?" or "have you seen feature 
X in this car?" rather than "have you been at another dealership?" . On the 
other hand, to find out whether she considered 6 Bob may need to ask Alice 
if she has seen the cal' manufacturer web site demonstration of some feature, 
or offer to e-mail her the link, and by doing that she might be moved to 
consider Internet pricing. Hence, the statements/questions that Bob uses to 
elicit information about awareness may require a trade-off since they could 
provide Alice with valuable aspects of the situation and sometimes the exact 
aspect that Bob was hoping to exploit. 

A third aspect of the strategie dialogue is the order of statements made. 
We can illustrate this with Alice's strategie choices. If she is aware of both 
6 and 7 she would prefer to reveal 6 before the price negotiations begin 
since she could provide a price quote which sets the upper bound to her 
reservation price. But in doing so she would indicate that she is likely to 
have obtained, or be aware of, various financing opportunities. She may 
want Lo hide this inforrnal.ion unLil afLe Lhe price is set and befere Lhe 
financing arrangement is made, the reason being that the salesperson may 
be willing to lower the price if he believes there is a high probability he can 
have a substantial gain from the financing agreement. Alice would like to 
make both statements, however their relative piace along the dialogue may 
inftuence the final outcome. 

The next aspect involves perception of attributes for which there is lit- 
tie information. Alice might be aware that the dealership may have some 
incentive from the manufacturer, i.e., of 3. However, she may not know 
the size of this incentive. Bob would probably not be willing to supply this 
information, or may not have access to it (see the discussion below). If Alice 
claims that she is aware of such incentives for the dealership yet is unable to 
provide any description of their magnitude, or some supporting argument, 
her statement is less credible and Bob may rightly deduce that she is only 
aware of the possibility but has no reason to assign high probability to a 
substantial impact on the dealership's costs. If credibility of one statement 
affects that of another, there is a cost associated with statements that are 
meaningful but not specific enough. 
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We note that Bob may not be informed of the dealership's actual costs 
if there is a concern that he might reveal it to potential buyers. In many 
cases cal' salespeople need the authorization of supervisors to make any sig- 
nificant price reductions along the bargaining process, this procedure allows 
managers to supervise the bargaining, but also to make pricing decisions 
based on information that may not be readily available to the salesperson. 
iVIoreover, Alice's awareness of 8 would lead her to treat the game differ- 
ently, as the solutions to bargaining with an intermediary could differ from 
direct negotiations. This also applies to Alice's perception of the incentives 
of the salesperson and could explain why firms tend to prominently state 
when their salespeople are not werking for commissions. 

As we noted in the previous section, an action can also yield a change of 
perception from both operational reasons and by deduction. For example, 
Bob may ask for Alice's driver license before a test drive, this might allow 
him to retrieve information about Alice that she is not aware he might 
be able to obtain. On the other hand, it could also alert Alice that her 
financial background could have impact on the financing offered and lead 
her to consider other lenders-7. 

Finally, a combination of higher order reasoning about perception may 
interact with strategic meaningful talk. For example, Alice may consider 3 
and even assume that stating it could be reliable, for example, she might 
have read that dealers are about to obtain delivery of a new model and have 
an incentive to deal' lot space. She mayalso be aware of 2-the existence of 
a formal invoice price, but she may not know its exact value. Bob may not 
realize that Alice is aware of 3. As such, Bob might be inclined to reveal the 
invoice price Lo Allee if he percel ves that she will model Lhe reserval.ion price 
of the dealership as no less than this invoice price. If Alice perceives that 
Bob is modeling her as unaware of 3 then Alice may decide to mention that 
she knows about invoice and try to induce Bob to show her documentation 
of this price, obviously making no mention of 3. Hence, Alice can obtain 
valuable information by reasoning about how Bob models her awareness 
and choosing her statements in the corresponding manner. Once she has 
the invoice price, Alice can more accurately estimate the dealership's true 
costs. 


4 Summary 


We have set out to present communication in games where the content 
of messages impacts strategic behavior without resorting to ex-ante as- 
sumed conventions. Our main observation is that in the case of games 
with unawareness the content of a message can change the perception of 
the game being played leading to changes in strategic behavior. Potentially 
the strongest form of meaningful talk we identified was when Bob told Alice 
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something about the situation that she already thought about, but that she 
did not realize Bob is also able to consider. This had led us to various new 
strategic considerations when the interpretation of the message is not only 
driven from a convention, or an arbitrary assignment of interpretation as in 
cheap talk, but is also derived from the expected change in perception of 
what game is being played. 

We used an illustrative example to show how aspects such as the tim- 
ing and order of statements, indirect questions, deliberate withholding of 
information from agents, trade-offs between revelation and discovery of per- 
ceptions, and the discovery of high order perceptions, inftuence strategic 
choices of meaningful statements in a dialogue which determines the bar- 
gaining game perceptions and eventually its outcome. 

We conclude that in a strategic setting a statement becomes meaningful 
as it describes the extent of the speaker's reasoning. In addition, statements 
become tools for molding perceptions and questions become tools for discov- 
ery of perceptions by generating answers. The implication is that dialogues 
are strategically intricate exchanges, since, to adopt a pragmatic approach, 
a statement does not only depend on the context but defines the context 
and determines the relevant strategic situation. 
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Abstract 


A speaker wishes to persuade a listener to take a certain action. The 
conditions under which the request is justified, from the listener's 
point of view, depend on the state of the world, which is known only 
to the speaker. Each state is characterized by a set of statements 
from which the speaker chooses. A persuasion mie specifies which 
statements the listener finds persuasive. We study persuasion mies 
that maximize the probability that the listener accepts the request if 
and only if it is justified, given that the speaker maximizes the proba- 
bility that his request is accepted. We prove that there always exists 
a persuasion mie involving no randomization and that all optimal 
persuasion mies are ex-post optima!. We relate our analysis to the 
field of pragmatics. 


1 Introduction 


A persuasion situation involves an agent (the speakers who attempts to 
persuade another agent (the listeneT) to take a certain action. Whether 


* Originally appeared in Theoretical Economics 1(4), pp. 395-410, 2006. 
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or not the listener should accept the speaker's suggestion depends on in- 
formation possessed by the speaker. In such a situation, the speaker often 
presents hard evidence to support his position, but is restricted as to how 
many pieces of evidence he can present. This restrietion may be due either 
to time constraints or to limitations on the listener's capability to process 
information. Our purpose in this paper is to shed light on the rules that 
determine which of the facts, presented by the speaker, the listener will find 
persuasive. 

The topic of this paper is related to a field in linguistics called pragmat- 
ics, which explores the rules that determine how people interpret an utter- 
ance, made in the course of a conversation, beyond its literal content (see 
Grice, 1989). Grice suggested that the leading principle in the interpreta- 
tion of utterances is what he termed the "cooperative principle", according 
to which the interpretation of utterances in a regular conversation can be 
made on the assumption that the speaker and the listener have common 
interests. However, the cooperative principle does not appeal’ to be relevant 
in a persuasion situation in which the agents may have confticting interests. 

The following example ciarifies the distinction between the pragmatics 
of conversation and the pragmatics of persuasion: You are discussing the 
chances of each of two candidates in an upcoming election. The electorate 
consists of ten voters. Assume that the other person has access to the views 
of these ten voters. Imagine that he has just informed you that a, d, and 
9 support candidate A. If it is a friendly conversation, then you are most 
likely to think that he has selected three people who represent the views of 
the majority of the voters. Thus, you are likely to be persuaded that A is 
likely Lo win Lhe election. H, on Lhe oLher hand, independenLly of Lhe truth, 
the other person is trying to persuade you that A will win, you will find this 
very same statement to be a weak argument since you will suspect that he 
has intentionally selected three supporters of A. 

What governs the pragmatic rules of persuasion? We propose an ap- 
proach analogous to Grice's cooperative principle in which the pragmatic 
rules of persuasion are determined by a fictitious designer before the dis- 
course begins. These rules govern the speaker's choice of facts to present 
in the knowledge that the listener will interpret his statements according 
to these rules. The rules are structured by the designer to maximize the 
probability that the listener will make the "right" decision (from his point 
of view and given the "true" situation) on the basis of the information pro- 
vided to him by a self-interested speaker and subject to constraints on the 
amount of information that can be submitted to him by the speaker. 

We conduct our investigation within the narrow boundaries of a par- 
ticular model in which several assumptions admittedly play a critical role. 
Our analysis is faithful to economie tradition rather than to the method- 
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ology of Pragmatics. Nevertheless, we believe that the study conducted 
here demonstrates the potentialof research to find a uniform principle that 
guides individuals in interpreting statements in persuasion situations. 

This paper belongs to a research program in which we apply a game 
theoretical approach to issues in Pragmatics. Im (Glazer and Rubinstein, 
2001) we study an example of a debate situation involving two parties each 
of whorn tries to persuade a third party to accept his position. Even closer 
to this paper is (Glazer and Rubinstein, 2004), which analyzes a persuasion 
situation in which aftel' the speaker makes his case the listener can obtain 
partial information about the state ofthe world. Aftel' specifying our current 
model, we will compare it to the one in (Glazer and Rubinstein, 2004). 


2 The model 


A speaker wishes to persuade a listener to take a certain action. The lis- 
tener can either accept or reject the speaker's suggestion (there is no partial 
acceptance). Whether or not the listener should be persuaded depends on 
the state, which is an element in a set X. A set A C X consists of all the 
states in which the listener would wish to be persuaded (i.e. to accept the 
speaker's suggestion) if he knew the state, and the set R = X \ A consists 
of all the states in which the listener would wish to reject the speaker's re- 
quest. The listener's initial beliefs about the state are given by a probability 
measure p over X. Denote by Px the probability of state x. 

We assume that for every state x, there is a set of statements c(<) 
that the speaker can make. Let S = U,zexo(ax). The meaning of "making 
statement s" is to present proofthat the event 0 !(s) = {x |sE o(x)} has 
occurred. 

In state x the speaker can make one and only one of the statements 
in o(x). Thus, for example, if the speaker can choose between remaining 
silent, making the statement œ, making the statement l3, or making both 
statements, the set a(x) consists of the four elements silence, a, I, and œ^. 

To summarize, we model a persuacion problem as a four-tuple 
(X, A, p,o). We say that the persuasion problem is finite if X is finite. 
We refer to the pair (X, ø) as a signal struciure. 


Comment. We say that a signal structure (Y, e) is oecioric if Y is a product 
set, ie. Y = XkEKYk for some set K and some sets yx KEK, and the 
speaker in state x can make a statement concerning the value of one of the 
components ofx, that is, e(x) = {(k,v) IKE K and v - x}. 

One might think that we could make do by analyzing only vectoric signal 
structures. To see that this is not the case, let (X, 0) be a signal structure. 
Let (Y,e) be the vectoric signal structure with Y = {0,//s. Every state 
X E X can be represented by the vector y(x) E Y, which indicates the 
statements available at x, that is, cp(x)(s) = 1 if sE o(x) and Ootherwise. 
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However, the two structures are not equivalent. First, we allow for the 
possibility that two states have the same set of feasible statements. Second, 
and more importantly, in the corresponding vectoric structure the speaker 
in any state is able to show the value of the component that corresponds 
to any statement s. In other words, he is always able to prove whether s 
is available or not. In contrast, in our framework the fact that the speaker 
can make the statement s does not necessarily mean that he can make a 
statement that proves that s is not available. 


We have in mind a situation in which the speaker makes a statement and 
the listener must then either take the action a, thus accepting the speaker's 
position, or the action 7, thus rejecting it. A persuasion rule determines 
how the listener responds to each of the speaker's possible statements. We 
define a persuasioti rule f as a function f 1 S 3 [0,1]. The function 
f specifies the speaker's beliefs about how the listener will interpret each 
of his possible statements. The meaning of f(s) = q is that following a 
statement s, with probability q the listener is "persuaded" and chooses a, 
the speaker's favored action. We call a persuasion rule f determinisiic if 
f(s) E {0,1} for all sE S. 

We assume that the speaker wishes to maximize the probability that 
the listener is persuaded. Thus, given a state x, the speaker solves the 
problem maxseo(2) f(s). The value of the solution, denoted by a(f, x), is 
the maximal probability of acceptance that the speaker can induce in state 
x. For the case in which o(z) is infinite, the solution can be approached 
but is not attainable and therefore we define a(f,x) = SUPsco(x) F S). 

Given the assumption that the speaker maximizes the probability of 
acceptance, we define the (listener's) error probability jz,(J) in state x as 
follows: If x E A, then #s(f)= 1- a(f,x), and if x E R, then #z(f) = 
a(J,x). The error pTobability induced by the persuasion rule fis m(J) = 
Š rcx Pelle (J). Given a problem (X, A,p, 7), an optimal persuasion rule is 
one that minimizes m(J). 

Note that persuasion rules are evaluated according to the listener's in- 
terests while those of the speaker are ignored. In addition, we assume that 
all errors are treated symmetrically. Our analysis remains the same if we 
add a variabie c, for the (listener's) "costs" of an error in state x and define 
the objective function to minimize one x PaCabal f). 


Example 2.1 ("The majority of the facts supports my position"). There 
are five independent random variables, each of which takes the values 1 
and 0 each with probability 0.5. A realization of 1 means that the random 
variabie supports the speaker's position. The listener would like to accept 
the speaker's position if and only if at least three random variables take the 
value 1. Inthe process of persuasion, the speaker can present the realization 
of at most m random variables that support his position. 
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Formally, X = {(XI,"" xs) | Xk E {0,1} for all k}, A= {x In(x) > 3} 
where N(x) = Ð, £k, Px = gg for all X EX, and o(z) = {« lk C fk lxk = 
I} and |«| < m}. 

If m = 3, the optimal persuasion rule states that the listener is per- 
suaded if the speaker presents any three random variables that take the 
value 1. The more interesting case is m = 2. If the listener is persuaded 
by the presentation of any two random variables that support the speaker's 
position, then the error probability is ie. The persuasion rule according to 
which the listener is persuaded only by the speaker presenting a set of two 
"neighboring" random variables ({1,2}, {2,3}, {3,4}, or {4,5}) with the 
value | reduces the error probability to = (an error in favor of the speaker 
occurs in the four states in which exactly two neighboring random variables 
support the speaker's position and in the state (1,0,1,0,1) in which the 
speaker is not able to persuade the listener to support him even though he 
should). 

The two mechanisms above do not use lotteries. Can the listener do 
better by applying a random mechanism? What is the optimal mechanism 
in that case? We return to this example aftel' presenting some additional 


results. 


Comment. At this point, we wish to compare the cutrent model with 
the one studied in (Glazer and Rubinstein, 2004). Both models deal with 
a persuasion situation in which (a) the speaker attempts to persuade the 
listener to take a particular action and (b) only the speaker knows the state 
of the world and therefore whether or not the listener should accept the 
speaker's request. 

Unlike the cutrent model, the speaker in the previous model could first 
send an arbitrary message (cheap talk) to the listener. Aftel' receiving the 
message, the listener could ask the speaker to present some hard evidence 
to support his request. The state of the world in that model is a realization 
of two random variables and the listener is able to ask the speaker to reveal 
at most one of them. Thus, unlike the cutrent model, in which the speaker 
simply decides which hard evidence to present, in the previous model the 
speaker has to "follow the listener's instructions" and the listener can apply 
a random device to determine which hard evidence he asks the speaker to 
present. That randomization was shown to often be a critical element in the 
listener's optimal persuasion rule (a point further discussed below). On the 
other hand, in the previous model we do not allow randomization during 
the stage in which the listener finally decides whether or not to accept the 
speaker's request, which we do allow in the current model. Allowing for 
such randomization in the previous model, however, is not beneficial to the 
listener, as we show to be the case in the current paper as weil. 


126 J. Glazer, A. Rubinstein 


The randomization in the previous paper is employed during the stage 
in which the listener has to decide which hard evidence to request from the 
speaker. Note that if in that model we restriet attention to deterministic 
persuasion rules, then it is a special case of the cutrent model. Eliminating 
randomization on the part of the listener in order to verify the information 
presented by the speaker, allows us to think about the persuasion situation 
in the previous model as one in which the speaker chooses which hard evi- 
dence to present rather than one in which the listener chooses which hard 
evidence to request. 

Randomization plays such an important role in the previous model be- 
cause it is, in fact, employed as a verification device. Without randomiza- 
tion, there is no value to the speaker's message since he could be lying. The 
listener uses randomization to induce the speaker to transfer more informa- 
tion than the information that is eventual!y verified. 

Although the cutrent model draws some inspiration from the previous 
one, the two papers relate to different persuasion situations and the results 
of the current paper cannot be derived from those of the previous one. 


3 Two lemmas 


We now present two lemmas that are useful in deriving an optimal persua- 
sion rule. 


3.1 A finite number of persuasive statements is sufficient 


Our first observation is rather technical though simpie. We show that if the 
set of states X is finite then even if the set of statements S is infinite there is 
an optimal persuasion rule in which at most IXI statements are persuasive 
with positive probability. 


Lemma 3.1. Let (X, A, p, c) be a finite persuasion problem. 
1. An optimal persuasion rule exists. 


2. There is an optimal persuasion rule in which {s | f(s) > O} does not 
contain more than IXI elements. 


Proo]. Consider a partition of S such that s and si are in the same cel! of 
the partition if 0 ~!(s) = o 1(s’). This partition is finite. Let T be a set of 
statements consisting of one statement from each cel! of the partition. We 
now show that for every persuasion rule 1, there is a persuasion rule 9 that 
takes a positive value only on T, such that a(g,x) = a(f,x) for al! x and 
thus m(g) = m(J). 

For every SÆT let Ss be the cel! in the partition of S that contains s. 
Define g(s) = sups/ess f(SI). For every s T define g(s) = 0. 
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For every state x, 


a(g,z)= max g(s) max sup f(s’) sup f(s’) =a(f,z). 

s€TNo(z) s€TNa(z) ses, s/Ea(x) 

Thus, we can confine ourselves to persuasion rules that take the value 0 
for any statement besides those in the finite set T. Any such persuasion rule 
is characterized by a vector in the compact set [0,1]T. The error probability 
is a continuous function on this space and thus there is an optimal persuasion 
rule j* with j*(s) = O for all s ¢ T. 

For every xE Slet s(x) E a(x) be a solution of maxseg(q) j*(5). Let g* 
be a persuasion rule such that 


Pod f*(s) ifs = at) for some x 
0 otherwise. 


The persuasion rule g* is optimal as weil since a(g*,x) = a(f*,x) for all 
x and thus m(g*) = m(J*). Thus, we can confine ourselves to persuasion 
rules for which the number of statements that persuade the listener with 
positive probability is no larger than the size of the state space. Q.E.D. 


3.2 The "L-principle" 
The following result is based on an idea discussed in (Glazer and Rubinstein, 
2004). 

Let (X, A,p, 7) be a persuasion problem such that for all x E X, o(x) 
is finite. We say that a pair (X, T), where XE A and T CR, is an L if for 
any s E o(x) there is t € T such that sE ø(t). That is, an L consists of an 
element x in A and a set T of elements in R such that every statement that 
can be made by X can also be made by some mernber of T. An L, (X, T) is 
minimal if there is no T/ C T such that (x, TI) is an L. 


Lemma 3.2 (The L-Principle). Let (x, T) be an L in the persuasion prob- 
lem (X, A,p,o) and let f be a persuasion rule. Then D ic{ejuT ulf). 


Proof. Recall that jz (J) = 1- a(f,x) and for every t ET, (J) = &(f,t). 
Therefore, 


He(f) 2 palf) + max pe(S) 
tE[xfUT 


> Half) + gas ne) 
= Half) + oi ], x) = 1. 


Q.E.D. 
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The following example demonstrates how the L-principle can be used 
to verify that a certain persuasion rule is optima!. For any persuasion 
problem, the L-principle provides a lower bound on the probability of error 
that can be induced by a persuasion rule. Thus, if a particular persuasion 
rule induces a probability of error equal to a Iower bound derived from the 
L-principle, then one can conclude that this persuasion rule is optima!. 


Example 3.3 ("I have outperformed the population average"). Consider 
a situation in which a speaker wishes to persuade a listener that his average 
performance in two previous tasks was above the average performance of 
the population. Denote by x: the proportion of the population that per- 
formed worse than the speaker in the first task and by x2 the proportion of 
the population that performed worse than the speaker in the second task. 
The speaker wishes to persuade the listener that x: +x2 > 1. The speaker 
knows his relative performance in the two tasks (that is, he knows x: and 
x2) but can present details of his performance in only one of the tasks. 
We assume that the speaker's performances in the two tasks are uncorre- 
lated. Formally, the signal structure is vectoric with X = [0,1] x [0,1]; the 
probability measure p is uniform on X; and A= (X, x2) |x: +x > I}. 
Note that if a statement is interpreted by the listener based only on its 
content, i.e. by stating that his performance was above 4 in one of the tasks, 


the speaker persuades the listener and the probability of error is L, 
The following argument (borrowed from Glazer and Rubinstein, 2004) 
shows that there exists an optimal persuasion rule according to which the 
listener is persuaded by the speaker if and only if the speaker can show that 
his performance in one of the two tasks was above 2. Furthermore, the 
minimal probability of error is +. 
A minimal L in this case is any pair (x, fy, Z}) where x E A, y, z E R, 

XI = Yl, and x2 = 22. 
The set 77 = ((XLX2) EA Ix < 2 and x2 < 2} is one of the three 
triangles denoted in 1 by the number 1. Any three points x = (X1, x2) E 71, 
= (Xi - 4,22) E Rand z = (XL X - +) E R establish an L. Ey the 
L-principle, for any persuasion rule f we have us( f) + My(f) + uz(f) > 1. 
The collection of all these L's is a set of disjoint sets whose union is the three 
triangles denoted in the figure by the number 1. Therefore, the integral of 
[+z (J) over these three triangles must be at least the size of 77, namely iB: 
Similar considerations regarding the three triangles denoted by the number 
2 and the three triangles denoted by the number 3 imply that the minimal 
error probability is at least 4. This error probability is attained by the 
persuasion rule according to which the listener is persuaded if and only if 


the speaker shows that either xz or x2 take a value of at least 2. 
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FIGURE 1. An optimal persuasion rule for 3.3. 


4 Randomization is not needed 


The next question to be addressed is whether randomization has any role in 
the design of the optimal persuasion rule. In other words, can the listener 
ever do better by making the speaker uncertain about the consequences 
of his statement? Glazer and Rubinstein (2004) show that in persuasion 
situations in which the listener can acquire partial information about the 
state of the world, uncertainty regarding what information he will acquire 
can be a useful device to the listener. However, as stated in 4.1 below, 
uncertainty is not useful to the listener in the present context. 


Proposition 4.1. 


1. For every finite persuasion problem (X, A, p,o), there exists an opti- 
mal persuasion rule f that is deterministic. 


2. For every persuasion problem (X, A, p, 7) and every c > 0, there exists 
a deterministic persuasion rule f* such that m(J*) < inff m(J) +s. 


Proo]. (1) Ey 3.1, there exists an optimal persuasion rule with a finite 
number of statements that induces acceptance with positive probability. 
Consider an optimal persuasion rule f with the fewest non-integer values. 
Let 0 <a, <... < o« < 1 be the values of f that are not 0 or 1. We show 
that K = 0. If not, consider the set T = {s I f(s) = a}. Let Y be the set 
of all states in which it is optimal for the speaker to make a statement from 
T, that is, Y = {x la(f,x) = ay}. 
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If the probability of Y NA is at least that of Y N R, then consider j+ 
which is a revision of j : 


J+(8) = ce for all s ET and j+(S) = j(8) for s T. 


Thus, a(j+,x) = œ for x E Y and a(j+,x) = a(f,x) for x ¢ Y. It 
follows that m(j+) < m(j). 

If the probability of Y N A is at most that of Y N R, then consider F. 
which is a revision of j: 


j-(8) = 0 for all s ET and j-(8) = j(8) for s ¢ T. 


Thus, a(j-,x) = 0 for x E Y and a(j-,x) = a(f,x) for x ¢ Y. It 
follows that m(j-) < m(j). 

The number of non-integer values used by either j+ or F; is reduced by 
1, which contradiets the assumption that j uses the the minimal number of 
non-integer values. 

(2) Let f’ be a persuasion rule such that m(f’) < infj m(j) +¢/2. Let 
n be an integer such that //n < £€/2. Let f” be the persuasion rule defined 
by f”(s) = max{m/n | mln < f’(s)}. Obviously m( f”) < m(f’) + n. 
The persuasion rule f” involves a finite number of values. By the proof of 
4.1 there is a deterministic persuasion rule /* with m(j*) < m(j"). Thus, 
mj*) < m( f’) +c/2 <infl m(j) +s. QED. 


Example 4.2 (Example 2.1 revisited: a Solution). We return now to ex- 
ample 2.1 and show that no persuasion rule induces a probability of er- 
ror less than =: Consider an optimal persuasion rule that is determin- 
istic. Thus, fg is either O or 1 for any state x. Ey the L-principle, 
#4(4,1,1,0,0) + #(4,1,0,0,0) + #4(4,0,1,0,0) + #4(0,1,1,0,0) 2 1 and similar inequalities 
hold for any of the other 9 states in which exactly three aspects support the 
speaker. Summing up over these 10 inequalities yields 


XO pe +3 X u< 10, 


n(x)=3 n(x) =2 


Using the fact that Hg is either 0 or 1 implies that pa Ha tae Ma = 
4 and thus Foy Pahi = $. 

Let us now describe an optimal persuasion rule for this case. Partition 
the set of random variables into t he two sets {I, 2,3} and {4,5}. The listener 


is persuaded only if the speaker can show that two random variables from 


the same cell of the partition support him. In states in which there are at 
least three random variables in favor of the speaker, at least two of them 
must belong to the same cell and, thus, the speaker is justifiably able to 
persuade the listener. However, in the four states in which exactly two 


A Study in the Pragmatics of Persuasion 131 


random variables belonging to the same cell support the speaker's position, 
the speaker is able to persuade the listener even though he should not be 
able to. Thus, the probability of error under this persuasion rule is g 
This persuasion rule seems to be attractive when the partition of the 
random variables is prominent. For example, if the random variables are 
associated with Alice, Beth, Christina, Dan, and Edward, they can natu- 
rally be divided into two groups by gender. Given the constraint that the 
speaker cannot refer to more than two individuals, we have found an opti- 
mal persuasion rule whereby referring to two individuals of the same gender 


is more persuasive than referring to two individuals of different genders. 


Example 4.3 (Persuading someone that the median is above the expected 
value). A speaker wishes to persuade the listener that the median of the 
values of three independent random variables uniformly distributed over 
the interval [0,1] is above 0.5. The speaker can reveal the value of only one 
of the three random variables. Is it more persuasive to present a random 
variabie with a realization of 0.9 or one with a realization of 0.67 

Formally, let X = [0,1] x [0,1] x [0,1] with a uniform distribution and 
A = {(XL X2, X3) | two of the values are above 0.5}. Let x; = ti denote the 
statement "the realization of the variable Xi is t;” and S(t], t2, t3) = {Xl = 
ti, X2 = t2, X3 = t3}' Inother words (X, S) is vectoric. 

The persuasion rule according to which the listener is persuaded only by 
the statement XI = +: where t: > Ł yields a probability of error of H, We 
will employ the L-principle to show that this persuasion rule is optima!. 

Note that the space X is isomorphic to the probabilistic space Y x Z with 
a uniform distribution, where Y = (0, 4] x [0,4] x [0, 4] and Z = {-I, I} x 
{-I, I} x {-I, I}, by identifying a pair (y, Z) with X (4 + YiZi)i=1,2,3. 

As aresuit, every (y, (1,1,-1)) EA is part of an L with (y, (-1,1, -1)) E 
Rand (y, (1,-1, -1)) ER. 


Thus we obtain the following inequalities: 


Hadi + Hy(—1,1,-1) + Kaiz 1 
Ly, (,—1,1) + Hty,(-1,-1,1)) + M(y,,-1,-1)) 2 1 
My (1,1,1) + Hy, (=1,1,—1)) + M(y,(-1,-1,)) 2 1 


Hence 


Miy,(,1,-1) + PG,G,-1,)) + By, (-1,1,1)) F 
2M(y,(—1,1,-1)) + 2(y,(1,-1,-1)) + 2(y,(-1,-1,1)) 2 3. 


For deterministic persuasion rules it must be that at least two of the 


variables j1(y,z) take the value 1 and, thus, for all y, we have 2z Piy, z) ely, z) 2 
2 = E, If there exists a persuasion rule that yields an error probability 
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strictly less than E, then by 4.1(ii) there is also a deterministic persuasion 
rule that yields an error probability less than L, Thus, the persuasion rule 


described above (which yields an error probability of exactly +) is optima!. 


5 A procedure for finding an optimal persuasion rule 


We are now able to prove a proposition that reduces the task of finding an 
optimal persuasion rule to a simple optimization problem. 


Proposition 5.1. Let (X, A,p,o) be a finite persuasion problem. Let 
(ue )zex be a solution to the optimization problem 


min Dalle S-t. fg E {O, 1} for all x E X, and 


{He Ex £ 


u«> 1 for any minimal L, (x, T). 
tE{x}ur 


Then there is an optimal persuasion rule that induces the probabilities of 
errors (JH) xEX. 


Proo]. Ey 4.1 we can restriet ourselves to deterministic mechanisms. Ey 
3.2 any persuasion rule satisfies the constraints (regarding the L's), so it 
is sufheient to construct a persuasion rule f that induces the optimal error 
probabilities vector (x )sex. 

Define f(s) = 1 for any signal s such that there exist X E A with s E o(z) 
so that p% = 0 and py = 1 for all YE R with sE oly). Define f(s) = 0 for 
any other signal s. 

It is sufficient to show that for all x, the induced probability wz(f) < px. 
Let XE A and u% = 0. There is a statement Sx E o(x) so that p} = 1 for 
all YE R such that Sx E o(y). Otherwise, there is an L, (x, T) such that 
Vrefajur Ht = 0. Thus f(sx) = 1 and He(f) = 0 

Let XE Rand pu} = 0. Then there is no s E ø(x) such that f(s) = 1 
and thus a(f, x) = 0 and Half) = už. QED. 


6 Ex-post optimality 
So far we have assumed that the listener is committed to a persuasion rule. 
In what follows, we address the question of whether the listener's optimal 
persuasion rule is one that he would indeed follow were he able to reconsider 
his commitment aftel' the speaker has made his statement. 

To motivate this analysis consider the following example. 


Example 6.1. The listener wishes to choose a guest for a TV news pro- 
gram. He is looking for a person with strong views about the issues of the 
day. There is a potential candidate who the listener knows is one of four 
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types: "hawk" (H), "dove" (D), a "pretender" (M) who can pretend to be 
either a hawk or a dove, or "ignorant" (J). The listener is not interested in 
the candidate's political views, but only in whether he has clear views one 
way or the other, i.e., if he is type H or D. The probabilities of the types 
are p(H) = p(D) = 0.2 and p(M) = p(J) = 0.3. 

The listener can interview the candidate, aftel' which he must decide 
whether or not to invite him onto the show. During the interview the 
listener plans to ask the speaker to make a statement regarding his views 
on cutrent issues. Assume that apart from remaining silent (action 0), type 
H can make only the statement h; D can make only the statement d; and 
M can make either statement hor d. Type I can only remain silent. Thus, 
o(H) = [h,O}, o(D) = {d,O}, o(M) = {h,d,O}, and o(1) = {0}. 

A "naive" approach to this problem is the following: Given the state- 
ment s, the listener excludes the types that cannot make the statement s 
and makes the optimal decision given the probabilities. For example, the 
message d excludes types land Hand therefore implies that the conditional 
probability that the speaker is of type D is 0.4. The listener thus rejects 
the speaker. This approach yields a probability of error of 0.4. 

Suppose that the listener can commit to how he will respond to the 
speaker's statement. It is easy to see that, in this example, the listener can 
reduce the probability of error to 0.3. The best persuasion rule is to invite 
the speaker to the show if and only if he makes the statement dor h. (This 
avoids the possibility that / is invited to the show but leaves the possibility 
that, in addition to Hand D, M might be invited.) 

Assume now that the listener is released from his commitment once a 
statement has been made. If he believes that M’s strategy is Lu uLLer d, 
then the listener, upon hearing the statement d, should attribute a higher 
probability to the possibility that he is facing M than to the possibility 
that he is facing D. Therefore, in this case he should not follow the optimal 
persuasion rule and should reject the speaker if he makes the statement d. 
H, however, the listener believes that M randomizes with equal probability 
between uttering d and h, then the listener, upon hearing the message d 
(h), should attribute the probability 4 to the possibility that he is facing 
type D (H) and, thus, should not deviate from the optimal persuasion rule. 


Note that the ex-post optimality of the optimal persuasion rule in this 
example hinges on the knife-edge condition that the speaker of type M ran- 
domizes with equal probability between hand d. This observation hints at 
the possibility that a persuasion problem might exist in which the listener's 
optimal persuasion rule is not ex-post optima!. However, as the analysis 
below demonstrates, this is never the case for finite persuasion problems . 

For a given persuasion problem (X, A, p, 7), consider the corresponding 
extensive persuasion game r(X,A,p,0). First, nature chooses the state 
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according to p; the speaker is then informed of the state x and makes a 
statement from the set o(«); and finally, aftel' hearing the speaker's state- 
ment, the listener chooses between a and 7. The payoff for the speaker is 
1 if the listener takes the action a and 0 otherwise. The payoff for the lis- 
tener is 1 if x E A and the action a is taken or if x E Rand the action r is 
taken, and 0 otherwise. We say that a certain persuasion rule f is credsble if 
there exists a sequential equilibrium of f(X, A, p, a), such that the listener's 
strategy is f. 


Example 6.2 (Example 3.3 revisited). The optimal persuasion rule de- 
scribed above is credible. The speaker's strategy of arguing in state (tl, (2) 
that xl = t: ift: > t: and that X2 = +: ift: > +: is optima!. The set of 
types that use the argument XI = t: is {(tl, X2) 1x2 < tı}. Conditionalon 
this set, the probability that (tl, X2) is in A is greater than 3 if and only if 
t-: > 2 and is less than 3 if and only if tr: < 2. 

Proposition 6.3. If the persuasion problem is finite, then any optimal 
persuasion rule is credible. 


This proposition follows from solving the auxiliary problem presented in 
the next section. 


Comment. The problem studied here can be viewed as a special case of 
a leader-follower problem in which the leader can commit to his future 
moves. As is weil known, it is generally not true that the solution to such 
an optimization problem is credible. We are not aware, however, of any 
general theorem or principle that addresses this issue and that can explain 
why it is the case that in our model the listener's optimal strategy is credible. 
This question remains for future research. 

We should emphasize, however, that 6.3 does not hold in case the listener 
has three actions, the speaker holds a fixed ordering over the actions, and 
the listener's preferences depend on the state. Consider, for example, the 
case in which the set of states is X = {I, 2}, the probability measure over 
X is p: = 0.4 and P2 = 0.6, the signal function is o(1) = {I}, o(2) = {I, 2}, 
and the listener's set of actions is {I, 2,3}. The speaker always prefers 1 
over 2 and 2 over 3 and the listener's utility from the state x and action a 
is u(1,1) = u(2,2) = 1, w(1,2) = w(2,1) = -1, and w(1,3) = u(2,3) = 0. 
The optimal persuasion rule for the listener is to respond to signal 2 with 
action 2 and to signall with action 3. However, once he observes signal 1 
it is better for the listener to take action 1. 


7 The bridges problem 


A group of individuals is partitioned into a finite number of types, which 
are members of a set X. The mass of type x is Px. Let S be a set of bridges 
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spanning a river. The individuals are loeated on one side of the river and 
would like to eross to the other side. Individuals of type x E X ean use only 
the bridges in the set o(a) Æ O. The set X is partitioned into two subsets, 
A whose members are welcome on the other side and R whose members are 
not. A deeision maker has to deeide, for eaeh bridge, the probability that 
that bridge will be open. The deeision maker eannot diseriminate between 
the individuals in A and R. Eaeh individual of type x ehooses a bridge in 
o(x) with the highest probability of being open from among the ones he ean 
use. The deeision maker's objeetive is to maximize the "net flow", i.e., the 
differenee in size between the group of type A's and the group of type R's 
erossing the river. 

A bridge poliey determines the probability with whieh eaeh bridge is 
open. A bridge poliey is eredible if there exists an assignment of types to 
bridges whereby: (i) eaeh type is assigned only to a bridge he ean use, 
(ii) within the set of bridges he ean use, eaeh type is assigned only to 
bridges with the highest probability of being open, and (iii) the mass of 
types in A who are assigned to a bridge that is open (closed) with strietly 
positive probability is at least as high (low) as the mass of types in R who 
are assigned to that bridge. We show that any optimal bridge poliey is 
eredible. 

Formally, a brulce policy is a function 0 :S — [0, 1] with the interpre- 
tation that O(s) is the probability that bridge s is open. Let a(O, x) = 
max{O(s) Is E o(x)}, that is the maximal probability of erossing the 
bridges that type x ean aehieve given the bridge poliey O. Let M(O) = 
aca Pa@(O, x) - J reRrPralO, x) be ealled the net flow. A bridge poliey 
is optimal ifiL maxiruizes N(O). Given a bridge policy O, a mtionalfeasible 
bridge assignment 3 is a function that assigns to eaeh type x a probability 
measure on g(x), sueh that ;3(x)(s) > 0 only for values of s that maximize 
O(s) in o(x). Given an assignment ĝ, the net assignment to bridge s is 
n(s,;3) = MeeaPs B(x) (s) — MaerP2B(x)(s). A bridge poliey 0 is credible 
if there is a rational feasible assignment ĝ sueh that for every s, O(s) >0 
implies n(s,;3) > 0 and O(s) < 1 implies n(5,;3) < 0. 


Claim 7.1. All optimal bridge polieies are eredible. 


Prooj]. Let 0* be an optimal bridge poliey. For any assignment £, let 


d(B) = In(s,;3)IO*(s) + n(s,;3)(1 - O*(s)). 
sE{sln(s,,6)<O} sE{sln(s,,6>O0} 


Let;3* be a minimizer of 6(B) over all rational feasible assignments. We show 
that 5(6*) = 0 and thus for all s sueh that O*(s) > 0 we have n/(s,;3) > 0 
and for all sueh s that O*(s) < 1 we have n(s, 3) <0. 
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Assume, for the purpose of contradiction, that 6(8*) > 0. Assume that 
there is a bridge S for which 0*(s) > 0 and n(5,;3*) < 0 (an analogous 
argument applies to the case in which there is a bridge Sfor which 0* (s) < 1 
and n(s,;3*) > 0). 

Let œ be the minimum of O*(s) over {s 1O*(s) > 0 and n(s,;3*) < O}. 
Let S(a) = {s 1O*(s) = a}. Let X(a) = {x 1;3*(x)(s) > 0 for a bridge s 
such that s E S(a)}, that is, X (æ) is the set of types who are assigned by;3* 
to the bridges whose probability of being open is œ. Note that types in X (a) 
cannot do better than trying to cross a bridge in S(a) and are indifferent 
between all bridges in S(a). Let Sa = {s E S{a) In(s,;3*) < O}. The set 
Sa is not empty and contains all bridges that are open with probability œ 
and for which the net assignment is negative. 

Let Yl, ...,in: be the longest sequence of distinct bridges in S(a)- Sa 
such that for every Yt, 


© a(Yt, B*) = 0 


(ii) there exist x E Rand Yo E Sau {Y1, ... ,ys-1} such that 8* (x )(yo) > 0 
and Yt E o(z). 


In other words, under ĝ* each Yt is a bridge with a zero net transfer such 
that there is a positive mass of types in R that can cross Yt and is assigned 
by 6* either to cross a bridge that precedes Yt in the sequence or to cross a 
bridge in Sa. 

Denote Z = Sau {Y1,..., Yr}. There are two possibilities: 


(i) There is no sE S(a)- Z, xE RP, and z E Z such that s E o(x) and 
B* (x)(Z) > 0. That is, there is no bridge S outside Z that is opened 
with probability œ and that can be crossed by a type in R who can 
cross the river with probability œ. The net transfer in Z is negative. 
Reducing the probability of transfer to all bridges in Z will increase 
the total net flow, thus violating the optimally of O*. 


(ii) There is s E S(a)- Z, x E R, and z E Z such that s E o(z) 
and ;3*(x)(z) > 0. By the definition of (Yl, ...,Yr) it must be that 
n(s, 8*) > 0. It follows that there are sequences of distinct bridges 
SO,SL ...,SK = s and types io,...,ik-1 E R such that sa ESa, 
3*ik)(Sk) > 0, and Sk+1 E o(ig) (for k = 0,...,K - 1). This allows 
us to construct a new rational assignment £ by shifting a positive 
mass of types in R from sa to si, from si to s2, and so on, such 
that (8) < 6(8*). Formally, let € be a positive number such that 
for k = 0,...,K -1 we have c < ;3*ik)(Sk), € < n(sK,;3*), and 
c < In(so,;3*)I. Define;3 as an assignment that is obtained from;3* by 
successively shifting to Sk+/ a mass c of individuals of type ik assigned 
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by;3* to cross sk. For all bridges with the exception of sa and sx we 
have n(s,;3) = n(s,;3*). Furthermore, n(sK,;3) = n(sk,;3*) - c >0 
and n(so,;3) = n(so, B*) +c <0. Thus, ô(8) = 6(8*)- œe- (1- ade, 
contradicting the choice of 6*. 


Thus, it follows that there exists a rational feasible assignment with 
nonnegative net flow on all open bridges and nonpositive net flow on all 
closed bridges. Q.E.D. 


8 Concluding remarks 


This paper has attempted to make a modest contribution to the growing 
literature linking economie theory to linguistics. Our purpose is not to 
suggest a general theory for the pragmatics of persuasion but rather to 
demonstrate a rationale for inferences in persuasion situations. 

One of our main findings is that any optimal persuasion rule is also ex- 
post optimal. It is quite rare that in a principal-agent problem the optimal 
incentive scheme is one that the principal would wish to obey even aftel' the 
agent has made his move. The bridge problem described in 7 provides an 
example of a principal-agent problem that in fact does have this property. 
The problem discussed in (Glazer and Rubinstein, 2004) is shown there to 
have this property as weil. The generalizability of this result is still an open 
question. 

Our work is related to several areas of research in linguistics and eco- 
nomics. In the linguistics literature, our paper belongs to the emerging 
field that tries to explain pragmatic rules by employing game theoretical 
methods. In our approach, pragmatic rules determine a game between the 
participants in the discourse. Whatever the process that created these rules, 
it is of interest to compare them with the rules that would have been chosen 
by a rational designer seeking to maximize the functionality of the discourse. 
Such an approach is suggested in (Glazer and Rubinstein, 2001, 2004) and 
discussed in (Rubinstein, 2000). A recent collection of articles in (Benz 
et al., 2006) presents various ideas that explain pragmatics phenomena us- 
ing game theoretical tools. 

Within the economie literature our paper relates to two areas of research. 

The first investigates sender-receiver games (see Crawford and Sobel, 
1982) in which one agent (the sender) sends a costless message to the other 
(the receiver). The receiver cannot verify any ofthe information sent by the 
sender and the interests of the sender and the receiver do not necessarily 
coincide. The typical question in this literature is whether an informative 
sequential equilibrium exists. 

The second (and closer) area of research studies models where a principal 
tries to elicit verifiabie information from the agent(s). The agent however 
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can choose which pieces of information to convey. Among the early pa- 
pers on this topic are (Townsend, 1979), (Green and Laffont, 1986), and 
(Milgrom and Roberts, 1986), and among the more recent are (Bull and 
Watson, 2004), (Deneekere and Severinov, 2003), (Fishman and Hagerty, 
1990), (Forges and Koessier, 2005), (Lipman and Seppi, 1995), and (Shin, 
1994). 
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These were her internal persuasions: "Old fashioned notions; country 
hospitality; we do not profess to give dinners; few people in Bath 
do; Lady Alicia never does; did not even ask her own sister's family, 
though they were here a month: and I dare say it would be very 
inconvenient to Mrs Musgrove; put her quite out of her way. I am 
sure she would rather not come; she cannot feel easy with us. I will 
ask them al! for an evening; that will be much better; that will be 
a novelty and a treat. They have not seen two such drawing rooms 
before. They will be delighted to come to-morrow evening. It shal! 
be a regular party, small, but most elegant." 


-Jane Austen, Persuasion 


Jacob Glazer and Ariel Rubinstein profter an exciting new approach to an- 
alyze persuasion. Perhaps even without being aware of it, and at least not 
acknowledged in the bibliography, their paper addresses questions that ar- 
gumentation theorists, logicians, and cognitive and social psychologists have 
been interested in since Aristotle's Rhetoric. Traditionally, argumentation 
was thought of as an activity involving knowiedge, beliefs, opinions, and it 
was contrasted with bargaining, negotiation and other strategie activities 
involving coereion, threats, deception, and what have you. More recently, 
however, several theorists have argued that strict boundaries are concep- 
tually indefensible and undesirable methodologically, separating as they do 
researchers who would more fruitfully combine efforts. Katia Sycara, for 
instanee, writes that "persuasive argumentation lies at the heart of nego- 
tiation" (Sycara, 1990), identifying various argumentation and negotiation 
techniques on the basis of careful empirical research on labor organizations. 
Simon Parsons, Catles Sierra, and Nick Jennings, by contrast, develop mod- 
els of argumentation-based negotiation (Parsons et al., 1998) with a high 
level of logical formality. Chris Provis, to mention a third, perhaps more 
skeptical representative, gives a systematic account of the distinction be- 
tween argumentation and negotiation suggesting to locate persuasion right 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 141-150. 
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in the middle (Provis, 2004).1 Glazer and Rubinstein's work enriches this 
literature with an analysis of persuasion. Using machinery from a formal 
theory of negotiation par excellence, economie theory, they develop a model 
of persuasion problems in which a speaker desires a listener to perform a 
certain action. Informing the listener about the state of the world is the only 
thing the speaker can do, but he can do it in many ways. By strategically 
making a statement that maximizes the likelihood that the listener decides 
to perform the action, the speaker exploits a peculiar feature of the model, 
namely, that the speaker does not need to teil the listener the whole truth; 
the truth alone suffices. Persuasion, for Glazer and Rubinstein, is telling the 
truth strategically, and phrased in the framework of a novel methodology 
this new approach merits close attention. 

A speaker, a listener, and a tuple (X, A, p, a) with X a set of worlds (not 
necessarily finite), A eX, p a probability measure over X, anda: X = S 
a function mapping worlds to sets of proposition (symbols?) in S, that is 
all there is to a "persuasion problem." There is a certain action that the 
speaker wants the listener to perform. The listener wants to perform it just 
in case the actual world is an element of A. The speaker, by contrast, wants 
the listener to perform the action even if the actual world is a mernber of the 
complement R of A. Since the listener does not have full information about 
the world but only "initial beliefs... given by a probability measure pover 
X” (Glazer and Rubenstein, 2008, p. 123), he is partly dependent on the 
speaker who has full knowledge of the world. Vet the speaker is under no 
obligation to report his full knowledge to the listener. The rules fixed by o 
allow the speaker to make all of the statements contained in o(2), if x is the 
acl.ual world. Clazer and Rubinstein wriLe that "Lhe rneaning of 'rnaking 
statement s' is to present proof that the event 0 1(s) = {xls E o(x)} has 
occurred" (page 5). Strategically picking such an s characterizes persuasion: 
an S with a large o !(S) will generally do better than a small one. The 
"persuasion function" f: S — [0,1], moreover, is intended to capture "the 
speaker's beliefs about how the listener will interpret each of his possible 
statements" (p. 124). The statement f(s) = q means that "following a 
statement s, there is a probability of q that the listener will be ‘persuaded’ 
and choose... the speaker's favored action" (p. 124). The speaker solves the 
maximization problem 


max f(s), 
s€a(a) 

where x is the actual world.é From the perspective of the listener, if the 

speaker makes a statement ¢ such that f(t) = MaXsco(w) f(s) there is a 

probability wz(f) that by using f he makes an error at x to perform an 


1 [ owe mueh to Chris Provis' exposition in (Provis, 2004). 
2 [f o(x) is infinite, the supremum of the expression can be approached. 
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action while x ¢ A, or not to perform an action at x while x E A. These 
probabilities are given by 


mip- [: Ke ifxEA 


hax setr) s otherwise. 


The listener chooses a persuasion rule that solves the minimization problem 


i x 
; pin dy Pl Juel f) 
given his beliefs p.? 


If this is a model, what does it model? Glazer and Rubinstein give sev- 
eral examples of persuasion problems. They bear names suggesting rather 
concrete applications ("The Majority of the Facts Support My Position," "I 
Have Outperformed the Population Average," ) as weil as technical, perhaps 
contrived, ones ("Persuading Someone that the Median is Above the Ex- 
pected Value"). In the first example, the speaker tosses a coin five times in 
a row, and wants the listener to perform a certain action the listener wants 
to perform just in case the coin landed heads at least three times. If the 
persuasion problem is such that the speaker can show how the coin landed 
in three of the five cases, he will of course succeed in persuading the listener, 
provided the coin landed heads at least three times. More interestingly, the 
speaker may only be able to reveal the outcomes of two coin tosses. Given 
that the listener only wants to perform the action in case the coin landed 
heads at least three times, there is always a risk involved in acting on the 
basis of the information the speaker provides to him. The listener may con- 
sider the persuasion rule according to which he performs the action just in 
case the speaker demonstrates that the coin landed heads twice. Among the 
total of 32 possible outcomes of the experiment (HHHHH, THHHH, and so 
on), there are 10 in which the coin landed heads twice, not thrice, and this 
makes the error probability of this rule poe The listener can improve if he 
adopts the persuasion rule to accept only if the coin landed heads twice in 
a row. This persuasion rules has error probability 5: 


An error in favor of the speaker will occur in the four states in which 
exactly two neighboring random variables [two successive coin tosses] 
support the speaker's position and in the state [HTHTH] in which 
the speaker will not be able to persuade the listener to support him 
even though he should. (p. 125) 


The example reveals a number of epistemic presuppositions behind the 
model Glazer and Rubinstein propose. Speaker and listener, for instance, 


3 Lemma | says that whatever the cardinality of S, there is always a solution to this 
minimization problem if the persuasion problem is finite. 
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have to know exactly what the rules of the game are. If the speaker does 
not know that he can reveal the outcomes of at most two successive coin 
tosses he will not consider the sequence in which the coin landed HTHTH 
as a problematic situation for him, and if the listener believes that the 
speaker may so be misinformed about the structure of the game he will also 
evaluate differently what is the optimal persuasion rule. One might even 
conjecture that as long as there is no common knowledge of the structure of 
the persuasion problem, game play is impossible. In addition, for the listener 
to calculate the error probabilities of the various persuasion rules he has to 
agree on the probability distribution of the relevant random variables. In 
the description of the formal model, Rubinstein and Glazer to that end 
insert a probability measure pover possible worlds with the initial beliefs of 
the listener as intended interpretation. The example, however, suggests that 
this probability is rather derived from the objective characteristics of set of 
possible worlds X, available to listener and speaker alike. Not only does 
the speaker, then, know what the actual world is in a situation in which the 
listener only has probabilistic beliefs concerning that issue, he also knows 
exactly what the listener believes about the world. 

Nor is this all. While strictly speaking no condition of possibility for an 
application of the model, Glazer and Rubinstein suggest that the speaker not 
only knows the listener's beliefs, but also the listener's prospective choice of 
strategy. Given the fact that the speaker has access to the listener's beliefs 
p, it is routine for him to calculate an optimal persuasion rule, and assuming 
that the listener is in some sense rational, the speaker is quite justified in 
believing that the listener will choose that rule. There is an interesting 
proviso, Lhough, Ior Lhe meaningfulness of Lhe definiLion of error probability 
depends not only on the fact that p expresses the listener's probabilistic 
beliefs concerning possible worlds, but also on the fact that the listener 
assumes that the speaker wants to maximize the likelihood that the listener 
perform the action. If the speaker did not want so to maximize, the listener 
would be unwise to build his risk estimation on the basis of the value of the 
solution to MaXscg(zx) fS). The speaker, for his derivation ofthe persuasion 
tule, needs to believe that the listener believes the speaker to be rational. 

For the speaker, it is quite deal' how to motivate the rule of rationality 
embodied in his maximizing the probability of acceptance. If the speaker 
has non-probabilistic beliefs concerning the persuasion rule f adopted by 
the listener, the only thing he needs to do is to piek a statement s, and 
it makes much sense to choose one that maximizes expected acceptance. 
Conceptions of rationality such as maximin or minimax regret are out of 
place here. For the listener this may be a bit different. The listener wants 
to piek a persuasion rule f: S — [0,1], and the most direct constraint is 
that f favors assigning high probability in cases in which the actual world is 
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an A-world, and low probability in cases in which it is an R-world. Without 
indication about how the elements from S relate to the A-ing or R-ing of the 
actual world, there is only one thing the listener could use to determine his 
strategy: his beliefs. If he believes with probability one that the world is in 
R, areasonabie persuasion rule assigns the value of zero (non-performance) 
to any statement made by the speaker. But the speaker knows what the 
actual world is, and the listener knows that the speaker knows it, so if the 
speaker makes a statement s with o !(s) C A to the effect that the world 
is definitely an A-world, then what should the listener do? This is a deal’ 
case of belief revision the model may not fully capture by assuming that the 
probabilities are objectively induced by the random variables determining 
X. For a rational choice of a persuasion rule the listener may not have 
enough information about the relation between the state of the world and 
the statement the speaker makes. 

A conditional statement can be made, though. If the listener believes 
that the speaker knows what persuasion rule f the listener chooses, and 
the listener believes that the speaker is rational, then the listener believes 
that in his calculation of the optimal persuasion rule he can use the pz(f) 
to assess the risk of making errors and solve min!: 510,1] X zex P(2) Me (Sf). 
Such a conditional statement, however, may delineate the applicability of 
the model in ways analogous to what we learn from the epistemic charac- 
terization of the Nash equilibrium (Aumann and Brandenburger, 1995). To 
constitute a Nash equilibrium, knowledge of strategies is presupposed. If 
I know what you are playing, and I am rational, and if you know what I 
am playing, and you are rational, then we will end up in a Nash equilib- 
rium. Such assurnptions are HoL always problernal.ic, Ior sure, but Lo justify 
making them requires in any case additional argumentation about, for in- 
stance, evolutionary (learning) mechanisrns or repeated game play. As the 
turn to iterative solution concepts constitutes to some extent an answer to 
the epistemic problerns with the Nash equilibrium, it may be interesting 
to investigate whether the model Glazer and Rubinstein put forward can 
similarly turn into the direction of common knowledge of game structure 
and rationality, especially if this can be accomplished in an extensive game 
framework. For consider the following dialog (election time in Italy): 


POLITICIAN: Vote for me. 

CITIZEN: Why? 

POLITICIAN: If you vote for me, 111 create one million new jobs. 
CITIZEN: That's unpersuasive. 

POLITICIAN: If you vote for me, 111 light for your freedom. 
CITIZEN: Persuaded. 
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This dialog, due to Isabella Poggi, illustrates a theory of persuasion in 
terrns of framing developed by Frederic Schick, among others (Poggi, 2005; 
Schick, 1988). While argumentation is about beliefs, and negotiation and 
bargaining are about desires and interests, persuasion, for Schick, involves 
the framing of options. A persuader persuades a persuadee by describing 
in novel and attractive terrns an action the persuadee found unattractive 
under previous description: 


We [may] want something under one description and... not want it 
under another. We may even want a proposition true and want a 
coreportive proposition false... 


Persuasion is the attempt to change a person's understanding of some- 
thing, to get him to see it in some way that prompts him to act as 
he would not have done. (Schick, 1988, p. 368) 


At first sight it may be too much to ask Rubinstein and Glazer to incorporate 
this insight, if an insight it is, in their formalism. Vet once we closely 
consider the way they set up the rules of the game, and in particular, the 
function they assign to function g, there are in fact two ways to recommend. 

The set o(x) contains exactly the statements that the speaker can make 
if the actual world happens to be x, and making a statement s amounts 
to demonstrating that the event o 1(s) = {xis E o(x)} has occurred. As 
aresuit, there is no room for the speaker to provide false information, 
but there is quite some room to provide true information tactically and 
strategically. A bit informally put, if o~1(s) = {xis E o(x)} contains many 
A-states and few R-states, then the speaker has good reasons to make the 
statement s, rather than another statement with less fortunate division 
between A and R.* From an extensional point of view, it suffices if 7 maps 
worlds to sets of worlds. Propositions, under this extensional perspective, 
are nothing more than sets of worlds. Extensionally speaking, modeling 
framing seerns pretty hopeless, though: a glass half full is the same as a 
glass half empty. From an intensional point of view, however, distinctions 
can be made between coextensive statements, and it is here that there is 
room for Glazer and Rubinstein to incorporate framing in their framework. 
The recipe is this. On the basis of a formal language, a set of statements 
S is defined from which the speaker may choose, and to make this set truly 
interesting, it has to be larger than p(X), the set of all statements possible 
with respect to X in purely extensional terms. To the description of the 
persuasion problem a relation of extensionality is added over the statements 
such that s =t iff a !(s) = o 1(t). Define a preference relation inside the 
resulting equivalence classes to express the listener's preferences for differing 


4 This is rough since it ignores the probabilistic judgments p the listener will invoke to 
calculate his error probability. 
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descriptions, and make the persuasion rule dependent on the statements in 
a systematic way described in terrns of the extensionality relation and the 
preferences of the speaker. 

A rather different approach to framing is possible, too, one that is much 
closer to the actual model Glazer and Rubinstein put forward. The speaker, 
in the persuasion problem as the authors define it, has a lot of freedom to 
choose a statement s to give the listener information about the actual world. 
The only restrietion is that the actual world be part of the set of worlds 
o~! (s). Instead of locating framing in intensionally different but extension- 
ally equivalent such sets, framing can also be modeled fully extensionally. 
Different statements s and ¢, each with the actual world in their g inverse 
image, frame the actual world differently, and one could very weil maintain 
that when the speaker selects what statement to make in Glazer and Ru- 
binstein's model, he is already engaged in framing decisions. While in the 
intensional solution to framing the speaker would choose between making 
a statement in terrns of the morning star and one in terms of the evening 
star, and opt for the latter because he knows that the listener is a night 
owl, in the alternative solution the speaker would describe the weather in 
terms of one of two extensionally different statements such as "the weather 
is good for sailing" and "the weather is good for kite surfing," depending 
on whether the listener likes sailing or kite surfing. 

The simple substitution of freedom for jobs in the dialog. and of drawing 
rooms for country hospitality in the quotation from Persuasion; is an exam- 
ple of persuasion by framing, however simple or simplistic such a switch may 
beo The dialog, and Austen's stream of consciousness avant la letire, point 
al, another important aspect of persuasiori, Loo: its temporal and sequenLial 
character. Argumentation theorists and logicians alike have noticed that 
the persuasive force one can exercise on others often depends on the order 
in which one presents one's arguments, offers, opinions, and threats. In 
dialogical logic, for instance, a proponent defends a proposition against an 
opponent who may attack according to clearly described rules. In its early 
days, dialogicallogic was used to promote intuitionist logic à la Heyting, or 
even to give it firm conceptual grounds. Contemporary dialogical logicians, 
however, see themselves engaged in building a "Third Way" alternative to 
syntactic and semantic investigations of logical consequence; or in one word, 
pragmatics. 

To get some feel for the kind of models used here, this is a dialog argu- 
ment to the effect that (p — Y) A vy) — w is a tautology, due to Rückert 
(2001): 


PROPONENT: ((y => 4Y) Ay) —> 4% 
OPPONENT: Weil, what if (p > Y) A y? 


PROPONENT: l'll show you W% in a minute. But wait, if you grant 
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(p — w) /\ ip, then I ask you to grant the left conjunct. 

OPPONENT: No problem, you get your ip > w 

PROPONENT: And what about the right conjunct? 

OPPONENT: That one you get, toa, ip. 

PROPONENT: Weil, if you say jp, 1 may say y to question you assuming 
the implication ip > %. 


OPPONENT: Right, I see what you're aiming at: you want me to say 
w, and I'll admit that w. 


PROPONENT: Perfect, that means I have shown you w in response to 
your initial query: ipse dixisti! 


Glazer and Rubinstein's approach to persuasion is decidedly static as it 
stands, but I believe that it can be turned dynamic at relatively low costs. A 
first step to consider is to take the probability distribution p as an expression 
of the truly subjective beliefs of the listener. This has the advantage that 
belief revision policies can be described to deal with cases in which the 
speaker comes up with new information, contradicting the listener's beliefs. 
In general, the listener may stubbornly stick to his p, but in more interesting 
persuasion problems the listener will revise his beliefs because, as it may be 
assumed, he knows that, however tactically and strategically the speaker will 
speak, he will at least speak the truth. Ina dynamic setting, furthermore, 
there may be more room for less heavy epistemic assumptions. To put it 
bluntly, my guess is that once persuasion games are represented as extensive 
games, common knowledge of game structure and rationality suffices to 
derive optimal persuasion rules. To my mind, this would constitute an 
increase in realism. 

An additional advantage is that extensive models can also take care of 
Aristotelian analyses of persuasion. Inthe Rheioric Aristotle distinguished 
three ways in which speakers can persuade their listeners. The rational 
structure of what the speaker says, the logos, first of all contributes to the 
persuasive force. Then the character of the speaker, his ethos, determines 
how credible and trustworthy the listener will judge the speaker, while, fi- 
nally, the emotional state of the listener, the pathos, plays a role in how a 
certain speech is received. Compare: a well-organized defense by a lawyer 
of established reputation in a law court with a serious and objective judge, 
with: a messy argument by a shabby lawyer directed at a judge involved 
in the case itself. And Aristotle is still highly popular, even among em- 
pirically oriented researchers. Isabella Poggi, for instance, agreeing with 
Schick about the role of framing in persuasion, sees expressions of ratio- 
nality, credibility, and emotionality as the modern analogs of Aristotle's 
tripartite division, and gives them all the force in her theory of persuasion 
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as hooking the speaker's goals to (higher) goals of the listener. In the Ital- 
ian dialog, for instanee, the speaker's goal that the listener votes for him 
was, first, hooked to diminishing unemployment. The goal of having a job, 
however, turned out not to be very important to the listener, and there- 
fore another goal was used, the more general one of freedom, of which the 
speaker had reason to believe that it would arouse the listener's emotions. 
At the end, the speaker in fact succeeded persuading (or so the story goes). 

Using the suggested extensional way of modeling framing, pathos can 
be captured by the preferenee relations the listener has over various de- 
scriptions of the actual world. Speaker's beliefs about such preferences can 
be included to describe specific persuasion strategies the speaker may wish 
to follow. The speaker is expected to try to describe the world in a way 
that makes it most attractive for the listener to perform the action but in 
order to be able to do that, the speaker needs to have some information 
concerning the listener's preferences.f Assumptions about general human 
preferences (concerning freedom, recognition, or what have you) make it 
possible for the speaker to do that without information about the specific 
listener. Ethos is captured by the belief revision policies of the listener. If 
the listener readily revises his beliefs upon hearing statements that contra- 
diet his own opinions, he reveals to trust the speaker showing the character 
of the speaker as a dependable person are at work. More skeptical belief 
revision policies, in all kinds of gradations, reveal the speaker's ethos to 
be functioning less than optimally. Extensive games can also model ways 
in which the speaker iteratively tries out reframing the description of the 
actual world. He may find out that the listener does not like sailing, so it 
does noL help hirn Lo describe Lhe world as one that is opt.irnal Ior sailing. In 
several models of persuasion, the listener's preferences play a crucial role. 
Logos, finally, gets modeled once speakers may take clever and less clever 
steps in iterative persuasion games, and it is especially here that cooper- 
ation with game theoretic approaches to logic (of which dialogical logic is 
only one among many) can be very fruitful (van Benthem, 2007). 
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Abstract 


We survey and discuss several solution concepts for infinite turn-based 
multiplayer games with qualitative (i.e., win-lose) objectives of the 
players. These games generalise in a natural way the common model 
of games in verification which are two-player, zero-sum games with 
w-regular winning conditions. The generalisation is in two directions: 
our games may have more than two players, and the objectives of the 
players need not be completely antagonistic. 

The notion of a Nash equilibrium is the classical solution concept 
in game theory. However, for games that extend over time, in par- 
ticular for games of infinite duration, Nash equilibria are not always 
satisfactory as a notion of rational behaviour. We therefore discuss 
variants of Nash equilibria such as subgame perfect equilibria and se- 
cure equilibria. We present criteria for the existence of Nash equilibria 
and subgame perfect equilibria in the case of arbitrarily many players 
and for the existence of secure equilibria in the two-player case. In 
the second part of this paper, we turn to algorithmic questions: For 
each of the solution concepts that we discuss, we present algorithms 
that decide the existence of a solution with certain requirements in a 
game with parity winning conditions. Since arbitrary w-regular win- 
ning conditions can be reduced to parity conditions, our algorithms 
are also applicable to games with arbitrary w-regular winning condi- 
tions. 


1 Introduction 


Infinite games in which two or more players take turns to move a token 
through a directed graph, tracing out an infinite path, have numerous ap- 
plications in computer science. The fundamental mathematical questions 


on such games concern the existence of optimal strategies for the players, 
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the complexity and structural properties of such strategies, and their real- 
isation by efficient algorithms. Which games are determined, in the sense 
that from each position, one of the players has a winning strategy? How to 
compute winning positions and optimal strategies? How much knowledge 
on the past of a play is necessary to determine an optimal next action? 
Which games are determined by memoryless strategies? And so on. 

The case of two-player, zero-sum games with perfect information and w- 
regular winning conditions has been extensively studied, since it is the basis 
of a rich methodology for the synthesis and verification of reactive systems. 
On the other side, other models of games, and in particular the case of 
infinite multiplayer games, are less understood and much more complicated 
than the two-player case. 

In this paper we discuss the advantages and disadvantages of several 
solution concepts for infinite multiplayer games. These are Nash equilibria, 
subgame perfect equilibria, and secure equilibria. We focus on turn-based 
games with perfect information and qualitative winning conditions, i.e., for 
each player, the outcome of a play is either win or lose. The games are not 
necessarily completely antagonistic, which means that a play may be won 
by several players or by none of them. 

Ofcourse, the world of infinite multiplayer games is much richel' than this 
class of games, and includes also concurrent games, stochastic games, games 
with various forms of imperfect or incomplete information, and games with 
quantitative objectives of the players. However, many of the phenomena 
that we wish to il!ustrate appeal' already in the setting studied here. To 
which extent our ideas and solutions can be catried over to other scenarios 
of infiniLe rnultiplayer gatnes is an interestlng topic of current research. 

The outline of this paper is as follows. Aftel' fixing our notation in 
Section 2, we proceed with the presentation of several solution concepts 
for infinite multiplayer games in Section 3. For each of the three solution 
concepts (Nash equilibria, subgame perfect equilibria, and secure equilibria) 
we discuss, we devise criteria for their existence. In particular, we wil! relate 
the existence of a solution to the determinacy of certain two-player zero-sum 
games. 

In Section 4, we turn to algorithmic questions, where we focus on games 
with parity winning conditions. We are interested in deciding the existence 
of a solution with certain requirements on the payoff. For Nash equilibria, it 
turns out that the problem is NP-complete, in genera!. However, there exists 
a natural restrietion of the problem where the complexity goes down to 
UPnco-UP (or even P for less complex winning conditions). Unfortunately, 
for subgame perfect equilibria we can only give an ExpTIIVIE upper bound 
for the complexity of the problem. For secure equilibria, we focus on two- 
player games. Depending on which requirement we impose on the payoff, we 
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show that the problem falls into one of the complexity classes UP N co-UP, 
NP, or co-NP. 


2 Infinite multiplayer games 


We consider here infinite turn-based multiplayer games on graphs with per- 
fect information and qualitative objectives for the players. The definition 
of such games readily generalises from the two-player case. A game is de- 
fined by an arena and by the winning conditions for the players. We usually 
assume that the winning condition for each player is given by a set of infi- 
nite sequences of colours (from a finite set of colours) and that the winning 
conditions of the players are, a priori, independent. 


Definition 2.1. An infinite (tum-based, qualitative) multiplayeT game is a 
tuple G = (I, V, (Vihen, E, X, (WinDiETI) where II is a finite set of plosjers, 
(V, E) is a (finite or infinite) directed graph, (V;);eq is a partition of V into 
the position sets for each player, X : V — C is a colouring of the position 
by some set C, which is usually assumed to be finite, and Win, C CW is the 
winning condition for player i. 


The structure G = (V,(V;)ien, E, X) is called the arena of g. For the 
sake of simplicity, we assume that ul} := {VE V : (u,v) E E} #0 for all 
u. E V, i.e., each vertex of G has at least one outgoing edge. We call Ga 
zero-sum game if the sets Win, define a partition of CW. 

A play of G is an infinite path through the graph (V, E), and a historu is 
a finite initial segment of a play. We say that a play zt is won by playeri E II 
if x(t) E Win.. The payoff of a play rt of G is the vector payt-r) E {O, 1}TI 
defined by pay(m); = 1 if zt is won by player i. A {pure} strategy of plosier i 
in G is a function g : V*V; — V assigning to each sequence XV of position 
ending in a position Vof player i a next position o(xv) such that (V, o(xv)) E 
E. We say that a play rr = m(O)a(1)... of G is consistent with a strategy 
o of player i if rij: +1) = o(n(O)... a(k)) for all k < w with m(k) E Vj. A 
strategy profile of G is a tuple (o;);em where o; is a strategy of player i. 

A strategy o of player i is called positional if o depends only on the 
current vertex, i.e if o(xv) = o(v) for all x E V* and v E V;. More 
generally, ø is called a finite-memcru strategy if the equivalence relation ~o 
on V* defined by X ~o X' if o(xz) = o(a’z) for all Z E V*V; has finite 
index. In other words, a finite-memory strategy is a strategy that can be 
implemented by a finite automaton with output. A strategy profile (o;)ien 
is called positional or a finite-memcru strategy profile if each o, is positional 
or a finite-memory strategy, respectively. 

It is sometimes convenient to designate an initial vertex Va E V of the 
game. We call the tuple (G,va) an initialised infinite multiplayeT game. A 
play (histOTY) of (G,va) is a play (history) of G starting with Va. A strategy 
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(strategy profile) of (G, va) is just a strategy (strategy profile) of Q A strat- 
egy o of some playeri in (G, va) is winning if every play of (G, va) consistent 
with ø is won by player i. A strategy profile (0;);c of (G, va) determines a 
unique play of (G, va) consistent with each o;, called the outcome of (0%)sen 
and denoted by ((0;)ser) or, in the case that the initial vertex is not under- 
stood from the context, ((o;):em)v)- Inthe following, we will often use the 
term game to denote an (initialised) infinite muliiplosjer game according to 
Definition 2.1. 

We have introduced winning conditions as abstract sets of infinite se- 
quences over the set of colours. In verification the winning conditions usually 
are ui-reqular sets specified by formulae of the logic SIS (monadic second- 
order logic on infinite words) or LTL (linear-time temporallogic) referring 
to unary predicates Pc indexed by the set C of colours. Special cases are 
the following well-studied winning conditions: 


- Biichi (given by F C C): defines the set of all œ E CW such that 
a(k) E F for infinitely many k < w. 


- co-Büchi (given by F C C): defines the set of all œ E CW such that 
a(k) E F for all but finitely many k < w. 


- Paritsj (given by a prioritij function Q : C — w): defines the set of all 
a E CW such that the least number occurring infinitely often in O(a) 
is even. 


- Rabin (given by a set Q of pairs (Gi, Ri) where Gi,Rj C C): defines 
the set of all œ E CW such that there exists an index i with a(k) E G; 
for infinitely many k < w but a(k) E R; only for finitely many k < w. 


- Sireett (given by a set Q of pairs (Gi, Ri) where Gi, R; C C): defines 
the set of all œ E CW such that for all indices i with a(k) E R; for 
infinitely many k < w also a(k) E O; for infinitely many k < w. 


- Muller (given by a family F of accepting sets F; C C): defines the set 
of all œ E CW such that there exists an index i with the set of colours 
seen infinitely often in œ being precisely the set Fi. 


Note that (co-)Biichi conditions are a special case of parity conditions 
with two priorities, and parity conditions are a special case of Rabin and 
Streett conditions, which are special cases of Muller conditions. Moreover, 
the complement of a Biichi or Rabin condition is a co-Biichi or Streett 
condition, respectively, and vice versa, whereas the class of parity conditions 
and the class of Muller conditions are closed under complement. Finally, 
any of these conditions is prefix independent, i.e., for every œ E CW and 
x E C* it is the case that œ satisfies the condition if and only if ca does. 
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We call a game G a multiplayeT ui-reqular, (co-)Biichi, paTity, Rabin, 
Streeti, OT Muller game if the winning condition of each player is of the 
specified type. This differs somewhat from the usual convention for two- 
player zero-sum games where a Biichi or Rabin game is a game where the 
winning condition of the first player is a Biichi or Rabin condition, respec- 
tively. 

Note that we do distinguish between colours and priorities. For two- 
player zero-sum parity games, one can identify them by choosing a finite 
subset of w as the set C of colours and defining the parity condition directly 
on the set C, i.e., the priority function of the first player is the identity 
function, and the priority function of the second player is the successor 
function k++ k +1. This gives paTity games as considered in the literature 
(Zielonka, 1998). 

The importance of the parity condition sterns from three facts: First, 
the condition is expressive enough to express any w-regular objective. More 
precisely, for every w-regular language of infinite words, there exists a deter- 
ministic word automaton with a parity acceptance condition that recognises 
this language. As demonstrated by Thomas (1995), this allows to reduce 
a two-player zero-sum game with an arbitrary w-regular winning condi- 
tion to a parity game. (See also Wolfgang Thomas’ contribution to this 
volume.) Second, two-player zero-sum parity games arise as the model- 
checking games for fixed-point logies, in particular the modal -calculus 
(Gradel, 2007). Third, the condition is simple enough to allow for posi- 
tional winning strategies (see above) (Emerson and Jutla, 1991; Mostowski, 
1991), i.e., if one player has a winning strategy in a parity game she also 
has a posit.ional one. In (Umrnels, 2006) iL was shown that Lhe lirst prop- 
erty extends to the multiplayer case: Any multiplayer game with w-regular 
winning conditions can be reduced to a game with parity winning condi- 
tions. Hence, in the algorithmic part of this paper, we will concentrate on 
multiplayer parity games. 


3 Solution concepts 


So far, the infinite games used in verification mostly are two-player games 
with win-lose conditions, i.e., each play is won by one player and lost by the 
other. The key concept for such games is deierminacsj: a game is determined 
if, from each initial position, one of the players has a winning strategy. 
While it is well-known that, on the basis of (a weak form of) the Axiom 
of Choice, non-determined games exist, the two-player win-lose games usu- 
ally encountered in computer science, in particular all w-regular games, are 
determined. Indeed, this is true for much more general games where the 
winning conditions are arbitrary (quasi-)Borel sets (Martin, 1975, 1990). 
In the case of a determined game, solving the game means to compute 
the winning regions and winning strategies for the two players. A famous 
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result due to Büchi and Landweber (1969) says that in the case of games 
on finite graphs and with w-regular winning conditions, we can effectively 
compute winning strategies that are realisabie by finite automata. 

When we move to multiplayer games andjor non-zero sum games, other 
solution concepts are needed. We will explain some of these concepts, in 
particular Nash equilibria, subgame perfect equilibria, and secure equilibria, 
and relate the existence of these equilibria (for the kind of infinite games 
studied here) to the determinacy of certain associated two-player games. 


3.1 Nash equilibria 


The most popular solution concept in classical game theory is the concept 
of a Nash equilibrium. Informally, a Nash equilibrium is a strategy profile 
from which no player has an incentive to deviate, if the other players stick 
to their strategies. A celebrated theorem by John Nash (1950) says that in 
any game where each player only has a finite collection of strategies there is 
at least one Nash equilibrium provided that the players can randomise over 
their strategies, i.e., choose mixed stmtegies rather than only pure ones. 
For turn-based (non-stochastic) games with qualitative winning conditions, 
mixed strategies play no relevant role. We define Nash equilibria just in the 
form needed here. 


Definition 3.1. A strategy profile (o;);en of a game (G,va) is called a 
Nash equilibriurn if for every player i EIland all her possible strategies 
o’ in (G, vo) the play (0’,(03);em\{}) is won by player i only if the play 
((@;)jen) is also won by her. 


It has been shown by Chatterjee et al. (2004b) that every multiplayer 
game with Borel winning conditions has a Nash equilibrium. We will prove 
a more general result below. 

Despite the importance and popularity of Nash equilibria, there are sev- 
eral probiems with this solution concept, in particular for games that extend 
over time. This is due to the fact that Nash equilibria do not take into ac- 
count the sequential nature of these games and its consequences. Aftel' any 
initial segment of a play, the players face a new situation and may change 
their strategies. Choices made because of a threat by the other players 
may no longer be rational, because the opponents have lost their power of 
retaliation in the remaining play. 


Example 3.2. Consider a two-player Büchi game with its arena depicted 
in Figure 1; round vertices are controlled by player 1; boxed vertices are 
controlled by player 2; each of the two players wins if and only if vertex 3 is 
visited (infinitely often); the initial vertex is 1. Intuitively, the only rational 
outcome of this game should be the play 1/23". However, the game has two 
Nash equilibria: 
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FIGURE 1. A two-player Büchi game. 


1. Player 1 moves from vertex 1 to vertex 2, and player 2 moves from 
vertex 2 to vertex 3. Hence, both players win. 


2. Player 1 moves from vertex 1 to vertex 4, and player 2 moves from 
vertex 2 to vertex 5. Hence, both players lose. 


The second equilibrium certainly does not describe rational behaviour. 
Indeed both players move according to a strategy that is always losing 
(whatever the other player does), and once player 1 has moved from vertex 1 
to vertex 2, then the rational behaviour of player 2 would be to change her 
strategy and move to vertex 3 instead of vertex 5 as this is then the only 
way for her to win. 


This example can be modified in many ways. Indeed we can construct 
games with Nash equilibria in which every player moves infinitely often 
according to a losing strategy, and only has a chance to win if she deviates 
from the equilibrium strategy. The following is an instructive example with 
quantitative objectives. 


Example 3.3. Let Qn be an n-player game with positions 0, ... ,n. Position 
n is the initial position, and position O is the terminal position. Player i 
moves at position i and has two options. Either she loops at position i (and 
stays in control) or moves to position i— I (handing control to the next 
player). For each player, the value of a play zr is (n + 1)/|xl. Hence, for all 
players, the shortest possible play has value 1, and all infinite plays have 
value O. Obviously, the rational behaviour for each player i is to move from 
i to i-1. This strategy profile, which is of course a Nash equilibrium, gives 
value 1 to all players. However, the 'most stupid' strategy profile, where 
each player loops forever at his position, i.e., moves forever according to a 
losing strategy, is also a Nash equilibrium. 
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3.2 Subgame perfect equilibria 


An equilibrium concept that respects the possibility of a player to change her 
strategy during a play is the notion of a subgame perfect equilibrium (Selten, 
1965). For being a subgame perfect equilibrium, a choice of strategies is not 
only required to be optimal for the initial vertex but for every possible initial 
history of the game (including histories not reachable in the equilibrium 
play). 

To define subgame perfect equilibria formal!y, we need the notion of a 
subgame: For a game G = (II, V,(V;)ien, E, X, (Wini)iETI) and a history h 
of Q, let the game G|, = (I, V,(Vi)ien, E, X, (Win, INiETI be defined by 
Win, h = {œ E CW: X(h) .œ E Win;}. For an initialised game (G, va) and 
a history hv of (G,vo), we cal! the initialised game (G|,,v) the subgame of 
(G,va) with history hv. For a strategy o of player i E II in Q, let oly, : 
V*V, = V be defined by g| (xv) = o(hxv). Obviously, ø|, is a strategy of 
player i in G|;,. 


Definition 3.4. A strategy profile (o;)jen of a game (G,va) is called a 
subgame perfect equilibrium (SPE) if (o;|n,)sen is a Nash equilibrium of 
(Gln, v) for every history hv of (G, vo). 


Example 3.5. Consider again the game described in Example 3.2. The 
Nash equilibrium where player 1 moves from vertex | to vertex 4 and player 2 
moves from vertex 2 to vertex 5 is not a subgame perfect equilibrium since 
moving from vertex 2 to vertex 5 is not optimal for player 2 aftel' the play has 
reached vertex 2. On the other hand, the Nash equilibrium where player 1 
moves from vertex | to vertex 2 and player 2 moves from vertex 2 to vertex 3 
is also a subgame perfect equilibrium. 


It is a classical result due to Kuhn (1953) that every finite game (i.e., 
every game played on a finite tree with payoffs attached to leaves) has a 
subgame perfect equilibrium. The first step in the analysis of subgame per- 
fect equilibria for infinite duration games is the notion of subgame-perfect 
determinacy. While the notion of subgame perfect equilibrium makes sense 
for more general classes of infinite games, the notion of subgame-perfect de- 
terminacy applies only to games with qualitative winning conditions (which 
is tacitly assumed from now on). 


Definition 3.6. A game (G, va) is subgame-perfect deiermined if there ex- 
ists a strategy profile (o;);cm such that for each history hv of the game one 
of the strategies o;|,, is a winning strategy in (G|p, v). 


Proposition 3.7. Let (G,va) be a qualitative zero-sum game such that 
every subgame is determined. Then (G, va) is subgame-perfect determined. 
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Prooj. Let (G, va) be a multiplayer game such that for every history hv there 
exists a strategy oł for some player i that is winning in (G|, V). (Note that 
we can assume that ar is independent of v.) We have to combine these 
strategies in an appropriate way to strategies as. (Let us point out that the 
trivial combination, namely o;(hv) := øł (v), does not work in genera!.) We 
say that a decomposition hv = hl . hz is good for player i w.r.t. vertex v if 
ol’ |n, is winning in (Gla, V). Ifthe strategy o” is winning in (Gp, v), then 
the decomposition h= h- € is good w.r.t. v, so a good decomposition exists. 

For each history hv, if oe is winning in (G|,,V), we choose the good 


(w.r.t. vertex v) decomposition h= A/h2 with minimal Al, and put 
oi(hv) : g” (hav). 


Otherwise, we set 
o;(hv) = oP (v). 

It remains to show that for each history hv of (G, va) the strategy o;|p, is 
winning in (G|;,, V) whenever the strategy af is. Hence, assume that ar is 
winning in (G|, v), and let z = (0)7(1)... be a play starting in 7(0) = v 
and consistent with o, lh’ We need to show that 1 is won by player i in 
(Gln; v). 

First, we claim that for each k < w there exists a decomposition of the 
form ka(0)... tti]: -1) = Al’ (hen(0)... rij:-1)) that is good for player i 
w.r.t. w(k). This is obviously true for k = 0. Now, for k > 0, assume that 
there exists a decomposition h7(0)... tti]: - 2) = Al (hon(0)... tti]: - 2)) 
that is good for player i w.r.t. rrij:- 1) and with hl being minima!. Then 
m(k) = oj(hn(0) a(k- D) = 0 (hon(0)...4(k- 1), and ha(0) ... r(k- 
1)= hy(hox(0) ij: -1)) is a decomposition that is good w.r.t. m(k). 


Now consider the sequence he, ht, of prefixes of the good decompo- 
sitions ha(O) ... tti]: - 1) hëh§r(0) tij- 1) (w.r.t. w(k)) with each 
ie being minima!. Then we have ho = ht > ..., Since for each k > 0 the 


decomposition ha(0) ... iJ: - 1) = h¥ *hS~‘x(0) ... rti]: - 1) is also good 
for player i w.r.t. m(k). As ~ is well-founded, there must exist k < w such 
that Al := k? = hi, and hz := kË = hi, for each k < I < w. Hence, we have 
that the play 7(k)a(k+1) ... is consistent with a Inom(0)...e(&—1)) Which is a 
winning strategy in (G|pm(o)...n(k—1),7(k)). So the play ha: is won by player i 
in (G, va), which implies that the play sis won by player i in (G|, V). Q.E.D. 


We say that a class of winning conditions is closed under taking sub- 
games, if for every condition X C Cw in the class, and every h E C%, also 
Xlh := {x E CW : hx E X} belongs to the class. Since Borel winning 
conditions are closed under taking subgames, it follows that any two-player 
zero-sum game with Borel winning condition is subgame-perfect determined. 
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Corollary 3.8. Let (G,va) be a two-player zero-sum Borel game. Then 
(G, va) is subgame-perfect determined. 


Multiplayer games are usually not zero-sum games. Indeed when we have 
many players the assumption that the winning conditions of the players form 
a partition of the set of plays is very restrictive and unnatural. We now drop 
this assumption and establish general conditions under which a multiplayer 
game admits a subgame perfect equilibrium. Im fact we will relate the 
existence of subgame perfect equilibria to the determinacy of associated 
two-player games. In particular, it will follow that every multiplayer game 
with Borel winning conditions has a subgame perfect equilibrium. 

In the rest of this subsection, we are only concerned with the existence 
of equilibria, not with their complexity. Thus, without loss of generality, 
we assume that the arena of the game under consideration is a tree or a 
fotest with the initial vertex as one of its roots. The justification for this 
assumption is that we can always replace the arena of an arbitrary game by 
its unravelling from the initial vertex, ending up in an equivalent game. 


Definition 3.9. Let G = (11, V, (Viken, E, X, (Wini)iETD be a multiplayer 
game (played on a forest), with winning conditions Win, C C’", The associ- 
ated class ZeroSum(G) of two-player zero-sum games is obtained as follows: 


1. For each player i, ZeroSum(G) contains the game Qi where player i 
plays Q with his winning condition Win., against the coalition of all 
other players, with winning condition CW \ Win.. 


2. Close the class under taking subgames (i.e., consider plays aftel’ initial 
histories). 


3. Close the class under taking subgraphs (i.e. admit deletion of posi- 
tions and moves). 


Note that the order in which the operations (1), (2), and (3) are applied 
has no effect on the class ZeroSum(C). 


Theorem 3.10. Let (G, va) be a multiplayer game such that every game in 
ZeroSum(G) is determined. Then (G, va) has a subgame perfect equilibrium. 


Proof. Let G = (11, V, (Vijen, E, X, (Wini)iETI) be a multiplayer game such 
that every game in ZeroSum(G) is determined. For each ordinal œ we define 
a set FE C E beginning with EO = E and 


E> N p“ 
acr 


for limit ordinals A. To define ®t! from E%, we consider for each player iE 
11 the two-player zero-sum game G7 = (V, Vi, E~, X, Win.) where player i 
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plays with his winning condition Win, against the coalition of all other 
players (with winning condition CW Wiu.). Every subgame of G7 belongs to 
ZeroSum(G) and is therefore determined. Hence we can use Proposition 3.7 
to fix a subgame perfect equilibrium (o0%,0%,;) of (G%,va) where of is a 
strategy of player i and o%, is a strategy of the coalition. Moreover, as the 
arena of G® is a forest, these strategies can be assumed to be positional. 
Let X{ be the set of all VE V such that of is winning in (Gf Ih, v) for the 
unique maximal history h of G leading to v. For vertices v E V; N X* we 
delete all outgoing edges except the one taken by the strategy o{, i.e., we 
define 


gett. py Uig E Bae Vin XË and v £o%(u)}. 
iETI 


Obviously, the sequence (/“),con is nonincreasing. Thus we can fix the 
least ordinal € with ES = E§&+! and define o; = of and al., = oF More- 
over, for each player j Æ% i let gj; be the positional strategy of player j in 
G that is induced by o_;. 

Intuitively, Player i's equilibrium strategy Ti is as follows: Player i plays 
o; as long as no other player deviates. Whenever some player j Æ i deviates 
from her equilibrium strategy o;, player i switches to g; j. Formally, define 
for each vertex v E V the player p(v) who has to be "punished" at vertex v 
where p(v) = L if nobody has to be punished. If the game has just started, 
no player should be punished. Thus we let 


pv) = L ifv is a root. 


At vertex v with predecessor u, the same player has to be punished as at 
vertex u. as long as the player whose turn it was at vertex u.did not deviate 
from her prescribed strategy. Thus for u. E V; and v E uE we let 


all if p(w) = L and v = o;(u), 
p(v) plu) if p(w) Ai, plu) A L and v = % pu) (u), 
i otherwise. 


Now, for each player i E II we can define the equilibrium strategy ri by 
setting 


cilv) if p(v) = L or p(v) = i, 
nlo) ieat otherwise 
for each vE V. 
It remains to show that (TiiETI is a subgame perfect equilibrium of 
(G,vo). First note that o; is winning in (G5 |n,v) if o% is winning in (GF |n, v) 
for some ordinal œ because if of is winning in (Gf Ih,v) every play of 
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(G2 |n, V) is consistent with ø% and therefore won by player i. As EE C 
E°~1, this also holds for every play of (g§ Th, v). Now let v be any vertex of G 
with h the unique maximal history of G leading to v. We claim that (T/)jETI 
is a Nash equilibrium of (G|z,V). Towards this, let T/ be any strategy of 
any player i E Il in Q; let z = ((T/jETIv, and let x’ = (TI, (T)jETIMi})V' 
We need to show that ha: is won by player i or that hm' is not won by 
player i. The claim is trivial if z = n’, Thus assume that rt Æ 7’ and fix 
the least k < w such that m(k +1) Æ a’(k +1). Clearly, a(k) E V; and 
t'(a(k)) Æ 7% (a(k)). Without loss of generality, let k = O. We distinguish 
the foliowing two cases: 


- 0; is winning in (Ge ln, v). By the definition of each Tj, zt is a play of 
(Gln, v). We claim that rt is consistent with aż; which implies that ha: 
is won by player i. Otherwise fix the least / < w such that m(l) E V; and 
oj(n(l)) A r(l +1). As o; is winning in (G5 |n;¥), o; is also winning 
in (GF |nm(o)...n 1) TD). But then (7(l),7(U+ 1) E ES\ EETL, a 
contradiction to BS = BETE, 


- o; is not winning in (Ge |n,v). Hence o_,; is winning in (Ge |n,v). As 
Tl(v) + Ti(v), player i has deviated, and it is the case that 7’ = 
(TI, (oji)jem q{i})o. We claim that 7’ is a play of (Gln, v). As oi 
is winning in (G5 Ih, v), this implies that An:'is not won by player i. 
Otherwise fix the least / < w such that (m'(l), m'(l +1) g ES to- 
gether with the ordinal œ such that (m'(1), m'(l + 1) E E®\ Bot!” 
Clearly, 7’(l) E V;. Thus o is winning in (G lrx (0)...w a- T (D), 
which implies that o, is winning in (G2 lnno). wa- TD). As 7’ is 
consistent with o_,, this means that o_; is not winning in (Ge |ns¥); a 
contradiction. 


It foliows that (T/)jETI = (TjIhjETI is a Nash equilibriurn of (G|, V) for every 
history hv of (G,vo). Hence, (TjjETI is a subgame perfect equilibrium of 
(G, Vo). Q.E.D. 


Corollary 3.11 (Ummels, 2006). Every multiplayer game with Borel win- 
ning conditions has a subgame perfect equilibrium. 


This generalises the result in (Chatterjee et al., 2004b) that every multi- 
player game with Borel winning conditions has a Nash equilibrium. Indeed, 
for the existence of Nash equilibria, a slightly weaker condition than the one 
in Theorem 3.10 suffices. Let ZeroSum(Ç)Nash be defined in the same way 
as ZeroSum(G) but without closure under subgraphs. 


Corollary 3.12. If every game in ZeroSum(C)Nash is determined, then G 
has a Nash equilibrium. 
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3.3 Secure equilibria 


The notion of a secure equilibrium introduced by Chatterjee et al. (2004a) 
tries to overcome another deficiency of Nash equilibria: one game may have 
many Nash equilibria with different payoffs and even several maximal ones 
w.r.t. to the componentwise partial ordering on payoffs. Hence, for the 
players it is not obvious which equilibrium to play. The idea of a secure 
equilibrium is that any rational deviation (i.e., a deviation that does not 
decrease the payoff of the player who deviates) will not only not increase the 
payoff of the player who deviates but it will also not decrease the payoff of 
any other player. Secure equilibria model rational behaviour if players not 
only attempt to maximise their own payoff but, as a secondary objective, 
also attempt to minimise their opponents' payoffs. 


Definition 3.13. A strategy profile (o;);en of a game (G, va) is called secure 
if for all players i Æ j and for each strategy g’ of player j it is the case that 


((ca)een) Z Win, or (Fijen o) E Win, 
= ((ai)ien) Z Win, or ((oi)en g} 7) E Win, . 


A strategy profile (o;)jen is a secure equilibrium if it is both a Nash equi- 
librium and secure. 


Example 3.14 (Chatterjee et al., 2004a). Consider another Biichi game 
played on the game graph depicted in Figure 1 by the two players 1 and 2 
where, again, round vertices are controlled by player 1 and square vertices 
are controlled by player 2. This time player 1 wins if vertex 3 is visited 
(infinitely often), and player 2 wins if vertex 3 or vertex 5 is visited (infinitely 
often). Again, the initial vertex is l. 

Up to equivalence, there are two different strategies for each player: 
Player 1 can choose to go from 1 to either 2 or 4 while player 2 can choose 
to go from 2 to either 3 or 5. Except for the strategy profile where player 1 
moves to 4 and player 2 moves to 3, all of the resulting profiles are Nash 
equilibria. However, the strategy profile where player 1 moves to 2 and 
player 2 moves to 3 is not secure: Player 2 can decrease player l's payoff by 
moving to 5 instead while her payoffremains the same (namely 1). Similarly, 
the strategy profile where player 1 moves to 2 and player 2 moves to 5 is not 
secure: Player | can decrease player 2's payoff by moving to 4 instead while 
her payoff remains the same (namely 0). Hence, the strategy profile where 
player 1 moves to 4 and player 2 moves to 5 is the only secure equilibrium 
of the game. 


It is an open question whether secure equilibria exist in arbitrary mul- 
tiplayer games with well-behaved winning conditions. However, for the 
case of only two players, it is not only known that there always exists a 
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secure equilibrium for games with well-behaved winning conditions, but a 
unique maximal secure equilibrium payoff w.r.t. the componentwise order- 
ing < on payoffs, i.e., there exists a secure equilibrium (0,7) such that 
pay((o’,7’)) < pay((o,7)) for every secure equilibrium (0’,TI) of (G, vo). 
Clearly, such an equilibrium is preferabie for both players. 

For two winning conditions Win.. Win, C V”, we say that the pair 
(Winj,Win-) is deiermined if any Boolean combination of Win, and Win, 
is determined, i.e., any two-player zero-sum game that has a Boolean com- 
bination of Win, and Win, as its winning condition is determined. 


Definition 3.15. A strategy o of player 1 (player 2) in a 2-player game 
(G, va) is strongly winning if it ensures a play with payoff (1, 0) (payoff (0, 1)) 
against any strategy 7 of player 2 (player 1). 

The strategy o is retaliating if it ensures a play with payoff (0,0), (1,0), 
or (1,1) against any strategy 7 of player 2 (player 1). 


Note that if (G,va) is a game with a determined pair (Win,, Win2) of 
winning conditions, then player | or 2 has a strongly winning strategy if 
and only if the other player does not have a retaliating strategy. 


Proposition 3.16. Let (G,va) be a two-player game with a determined pair 
(Winj, Win-) of winning conditions. Then precisely one of the following four 
cases holds: 


1. Player 1 has a strongly winning strategy; 
2. Player 2 has a strongly winning strategy; 
3. There is a pair of retaliating strategies with payoff (1, 1); 


4. There is a pair of retaliating strategies, and all pairs of retaliating 
strategies have payoff (0,0). 


Proo]. Note that if one player has a strongly winning strategy, then the 
other player neither has a strongly winning strategy nor a retaliating strat- 
egy. Vice versa, if one player has a retaliating strategy, then the other player 
cannot have a strongly winning strategy. iVIoreover, cases 3 and 4 exclude 
each other by definition. Hence, at most one of the four cases holds. 

Now, assume that neither of the cases 1-3 holds. In particular, no 
player has a strongly winning strategy. By determinacy, this implies that 
both players have retaliating strategies. Let (o,T) be any pair of retaliating 
strategies. As case 3 does not hold, at least one of the two players receives 
payoff 0. But as both players play retaliating strategies, this implies that 
both players receive payoff 0, so we are in case 4. Q.E.D. 


Infinite Multiplayer Games 165 


Theorem 3.17. Let (G, va) be a two-player game with a determined pair 
(Win- ,Win-) of winning conditions. Then there exists a unique maximal 
secure equilibrium payoff for (G, va). 


Proo]. We show that the claim holds in any of the four cases stated in 
Proposition 3.16: 


1. In the first case, player 1 has a strongly winning strategy o. Then, 
for any strategy T of player 2, the strategy profile (ø, T) is a secure 
equilibrium with payoff (1,0). We claim that (1,0) is the unique max- 
imal secure equilibrium payoff. Otherwise, there would exist a secure 
equilibrium with payoff 1 for player 2. But player 1 could decrease 
player 2's payoff while not decreasing her own payoff by playing g, a 
contradiction. 


2. The case that player 2 has a strongly winning strategy is analogous 
to the first case. 


3. In the third case, there is a pair (ø, T) of retaliating strategies with 
payoff (1,1). But then (ø, T) is a secure equilibrium, and (1,1) is the 
unique maximal secure equilibrium payoff. 


4. In the fourth case, there is a pair of retaliating strategies, and any 
pair of retaliating strategies has payoff (0,0). Then there exists a 
strategy o of player 1 that guarantees payoff 0 for player 2, since 
otherwise by determinacy there would exists a strategy for player 2 
that guarantees payoff 1 for player 2. This would be a retaliating 
strategy that guarantees payoff 1 for player 2, a contradiction to the 
assumption that all pairs of retaliating strategies have payoff (0,0). 
Symmetrically, there exists a strategy 7 of player 2 that guarantees 
payoff 0 for player 1. By the definition of g and 7, the strategy profile 
(0,7) is a Nash equilibrium. But it is also secure, since it gives each 
player the least possible payoff. Hence, (ø, T) is a secure equilibrium. 
Now assume there exists a secure equilibrium (o’, TI) with payoff (1, 0). 
Then also (o’,7) would give payoff 1 to player 1, a contradiction to 
the fact that (ø, T) is a Nash equilibrium. Symmetrically, there cannot 
exists a secure equilibrium (o’, TI) with payoff (0,1). Hence, either 
(0,0) or (1,1) is the unique maximal secure equilibrium payoff. Q.E.D. 


Since Borel winning conditions are closed under Boolean combinations, as 
a corollary we get the result by Chatterjee et al. that any two-player game 
with Borel winning conditions has a unique maximal secure equilibrium 
payoff. 


Corollary 3.18 (Chatterjee et al., 2004a). Let (G, va) be two-player game 
with Borel winning conditions. Then there exists a unique maximal secure 
equilibrium payoff for (G, va). 
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4 Algorithmic problems 


Previous research on algorithrns for multiplayer games has focused on com- 
puting some solution of the game, e.g., some Nash equilibrium (Chatterjee 
et al., 2aa4b). However, as we have seen, a game may not have a unique 
solution, so one might be interested not in any solution, but in a solution 
that fulfils certain requirements. For example, one might look for a solution 
where certain players win while certain other players lose. Or one might 
look for a maximal solution, i.e., a solution such that there does not exist 
another solution with a higher payoff. In the context of games with par- 
ity winning conditions, this motivation leads us to the following decision 
problem, which can be defined for any solution concept S: 


Given a multiplayer parity game (Q,va) played on a finite arena 
and thresholds Z,y E {O,/}k, deeide whether (G, va) has a solution 
(oijien E S(Q,va) sueh that % < pay ({(o%)sen)) < y. 


In particular, the solution concepts of Nash equilibria, subgame perfect equi- 
libria, and secure equilibria give rise to the decision problerns NE, SPE and 
SE, respectively. In the following three sections, we analyse the complexity 
of these three probiems. 


4.1 Nash equilibria 


Let (G, va) be a game with prefix-independent, determined winning condi- 
tions. Assume we have found a Nash equilibrium (0;);en of (G,vo) with 
payoff . Clearly, the play ((o;);en) never hits the winning region Wi of 
some player i with x; = @ because otherwise player i can improve her payoff 
by waiting until the token hits W; and then apply her winning strategy. 
The crucial observation is that this condition is also sufheient for a play to 
be induced by a Nash equilibrium, i.e., (G, va) has a Nash equilibrium with 
payoff @ if and only if there exists a play in (Q, va) with payoff Z that never 
hits the winning region of some player i with x, = a 


Lemma 4.1. Let (G,va) be a k-player game with prefix-independent, de- 
termined winning conditions, and let W; be the winning region of player i 
in Q There exists a Nash equilibrium of (G,va) with payoff x E fa,/}k 
if and only if there exists a play zt of (G,va) with payoff Z such that 
{n(k):k<w}n W = 0 for each player i with x; = a. 


Proo]. (=>) This direction follows from the argumentation above. 

(<) Let zt be a play with payoffx such that {7(k) :k < w}n W, = 0 for 
each player i with x; = a Moreover, let 7T_/ be an optimal strategy of the 
coalition 11 \ {j} in the two-player zero-sum game Qj where player j plays 
against all other players in Q and let 71; be the corresponding strategy of 
player i in G (where Tii is an arbitrary strategy). For each player iE 11, we 
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define a strategy o; in G as follows: 


o;(hv) i +1) if hv= (0)... t(k) < ez, 


T,j(hov) otherwise, 


where, in the latter case, h = hgp2 such that Al is the longest prefix of h 
still being a prefix of zt, and j is the player whose turn it was aftel' that 
prefix (i.e. hl ends in Vj), where j = i if hl = s. 

Let us show that (0;);eq is a Nash equilibrium of (G, va) with payoff Z. 
First observe that ((o; MIETI) = tt, which has payoff x, thus it remains to show 
that (o;);cn is a Nash equilibrium. So let us assume that some player iE II 
with x; = 0 can improve her payoff by playing according to some strategy g’ 
instead of aS. Then there exists k < w such that o’(a(k)) Æ o;(a(k)), and 
consequently from this point onwards ((0; ETI\ {i}, a") is consistent with 7-i, 
the optimal strategy ofthe coalition II\ {i} in gi. Hence, 7_; is not winning 
from m(k). Ey determinacy, this implies that 7(k) E Wj a contradiction. 

Q.E.D. 


As an immediate consequence, we get that the problem NE is in NP. 
However, in many cases, we can do better: For two payoff veetors %,y E 
{O, 1}k, let dist(x, Y) be the Hamming distance of & and Y, i.e., the num- 
be! $Ë; Ii - Ail of nonmatching bits. Jurdziriski (1998) showed that the 
problem of deciding whether a vertex is in the winning region for player 0 
in a two-player zero-sum parity game is in UP nco-UP. Recall that UP is 
the class of all problems decidabie by a nondeterministic Turing machine 
that runs in polynomial timp and has at most one accept.ing’ run on every 
input. We show that the complexity of NE goes down to UP n co-UP if the 
Hamming distance of the thresholds is bounded. If additionally the number 
of priorities is bounded, the complexity reduces further to P. 


Theorem 4.2 (Ummels, 2008). NE is in NP. If dist(Z%,%) is bounded, NE 
is in UP nco-UP. If additionally the number of priorities is bounded for 
each player, the problem is in P. 


Proo]. An NP algorithm for NE works as follows: On input (G,va), the 
algorithm starts by guessing a payoff zT < Z < y and the winning region 
W; of each player. Then, for each vertex v and each player i, the guess 
whether v E W; or v ¢ W; is verified by running the UP algorithm for the 
respective problem. If one guess was incorrect, the algorithm rejects im- 
mediately. Otherwise, the algorithm checks whether there exists a winning 
play from Va in the one-player game arising from G by merging the two play- 
ers, restricting the arena to G f nZi=a(V \ Wi), and imposing the winning 
condition Agni & A Aec =Q;, a Streett condition. If so, the algorithm 
accepts. Otherwise, the algorithm rejects. 
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The correctness of the algorithm follows from Lemma 4.1. For the com- 
plexity, note that deciding whether there exists a winning play in a one- 
player Streett game can be done in polynomial time (Emerson and Lei, 
1985). 

If dist(x, Y) is bounded, there is no need to guess the payoff Z. Instead, 
one can enumerate all of the constantly many payoffs Z < Z < y and check 
for each of them whether there exists a winning play in the respective one- 
player Streett game. If this is the case for some Z, the algorithm may 
accept. Otherwise it has to reject. This gives a UP algorithm for NE in 
the case that dist(x, y) is bounded. Analogously, a UP algorithm for the 
complementary problem would accept if for each Z there exists na winning 
play in the respective one-player Streett game. 

For parity games with a bounded number of priorities, winning regions 
can actually be computed in polynomial time (see, e.g., Zielonka, 1998). 
Thus, if additionally the number of priorities for each player is bounded, 
the guessing of the winning regions can be avoided as weil, so we end up 
with a deterministic polynomial-time algorithm. Q.E.D. 


It is a major open problem whether winning regions of parity games 
can be computed in polynomial time, in general. This would allow us to 
decide the problem NE in polynomial time for bounded dist(x, ¥) even if 
the number of priorities is unbounded. Recently, Jurdzinski et al. (2006) 
gave a deterministic subexponential algorithm for the problem. It follows 
that there is a deterministic subexponential algorithm for NE if dist(x, y) 
is bounded. 

Another line of research is to identify structural properties of graphs 
that allow for a polynomial-time algorithm for the parity game problem. 
It was shown that winning regions can be computed in polynomial time 
for parity games played on graphs of bounded DAG-width (Berwanger 
et al., 2006; Obdrzdlek, 2006) (and thus also for graphs of bounded tree 
width (Obdrzalek, 2003) or bounded entanglement (Berwanger and Grädel, 
2005)), and also for graphs of bounded clique width (Obdrzalek, 2007) or 
bounded Kelly width (Hunter and Kreutzer, 2007). It follows that NE can 
be decided in polynomial time for games on these graphs if also dist(x, 9) 
is bounded. 

Having shown that NE is in NP, the natural question that arises is 
whether NE is NP-complete. We answer this question affirmatively. Note 
that it is an open question whether the parity game problem is NP-complete. 
In fact, this is rather unlikely, since it would imply that NP = UP = 
co-UP = co-NP, and hence the polynomial hierarchy would collapse to its 
first level. As a matter of fact, we show NP-completeness even for the 
case of games with co-Biichi winning conditions, a class of games known 
to be solvable in polynomial time in the classical two-player zero-sum case. 
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Also, it suffices to require that only one distinguished player, say the first 
one, should win in the equilibrium. In essence, this shows that NE is a 
substantially harder problem than the problem of deciding the existence of 
a winning strategy for a certain player. 


Theorem 4.3 (Ummels, 2008). NE is NP-complete for co-Biichi games, 
even with the thresholds 7 = (1,0,...,0) and y= (1,...,1). 


Proo]. By Theorem 4.2, the problem is in NP. To show that the problem is 
NP-hard, we give a polynomial-time reduction from SAT. Given a Boolean 
formula y = Cl A... A Cm in CNF over variables XI, ...,X,, we build a 
game Go played by players 0,1, ...,n as follows. G, has vertices C/,’.. ,Cm 
controlled by player 0, and for each clause C and each literal X; or , Xi, a 
vertex (C, Xi) or (C, ,Xi), respectively, controlled by playeri. Additionally, 
there is a sink vertex L. There are edges from a clause Cj to each vertex 
(Cj, L) such that L occurs as a literal in Cj; and from there to CG mod m)+l. 
Additionally, there is an edge from each vertex (C,,Xi) to the sink vertex 
1. As Ll is a sink vertex, the only edge leaving | leads to L itself. For 
example, Figure 2 shows the essential part of the arena of G, for the formula 
p= (XIVX3V,X2) A (X3 VAX) A, X3. The co-Biichi winning conditions 
are as follows: 


- Player 0 wins if the sink vertex is visited only finitely often (or, equiv- 
alently, if it is not visited at all). 


- Playeri E {I...,} wins if each vertex (C, Xi) is visited only finitely 
often. 


Clearly, Gp can be constructed from y in polynomial time. We claim 
that y is satisfiable if and only if (G,,C1) has a Nash equilibrium where at 
least player O wins. 

(=) Assume that ¢ is satisfiable. We show that the positional strategy 
profile where at any time player 0 plays from a clause C to a (fixed) literal 
that satisfies this clause and each player j ~ 0 plays from ~X; to the sink 
if and only if the satisfying interpretation maps Xj to true is a Nash equi- 
librium where player 0 wins. First note that the induced play never reaches 
the sink and is therefore won by player O. Now consider any player i that 
loses the induced play, which can only happen if a vertex (C, Xi) is visited 
infinitely often. But, as player 0 plays according to the satisfying assign- 
ment, this means that no vertex (CI, ,X7) is ever visited, hence player i has 
no chance to improve her payoff by playing to the sink vertex. 

(<=) Assume that (G,,C)) has a Nash equilibrium where player 0 wins, 
hence the sink vertex is not reached in the induced play. Consider the 
variabie assignment that maps X; to true if some vertex (C, Xi) is visited 
infinitely often. We claim that this assignment satisfies the formula. To see 
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FIGURE 2. The game Gy for y = (X1 V X3 V 7X0) A (X3 V 7X4) A 2X3. 


this, consider any clause Cj . By the construction of Go, there exists a literal 
X; or ~X; in C; such that the vertex (Cj, Xi) or (Cj,,Xi), respectively, is 
visited infinitely often. Now assume that both a vertex (C, Xi) and a vertex 
(CI, ,Xi) are visited infinitely often. Then player i would lose, but could 
improve her payoff by playing from (CI, ,Xi) to the sink vertex. Hence, in 
any case the defined interpretation maps the literal to true thus satisfying 
the clause. Q.E.D. 


4.2 Subgame perfect equilibria 


For subgame perfect equilibria, we are not aware of a characterisation like 
the one in Lemma 4.1 for Nash equilibria. Therefore, our approach to 
solve SPE is entirely different from our approach to solve NE. Namely, we 
reduce SPE to the nonemptiness problem for tree automata (on infinite 
trees). However, this only gives an ExpTIIVIE upper bound for the problem 
as opposed to NP for the case of Nash equilibria. For the ful! proof of the 
following theorem, see (Ummels, 2006). 
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Theorem 4.4. The problem SPE is in ExpTIIVIE. If the number of players 
and priorities is bounded, the problem is in P. 


Proo] sketch. Without loss of generality, let us assume that the input game 
G is binaTY, i.e., every vertex of G has at most two successors. Then we can 
arrange all plays of (G,va) in an infinite binary tree with labels from the 
vertex set V. Given a strategy profile (o;);en of (G, vo), we enrich this tree 
with a second label component that takes the value 0 or 1 if the strategy 
profile prescribes going to the left or right successor, respectively. 

The algorithm works as follows: We construct two aliernaiing paTity tree 
automata. The first one checks whether some arbitrary tree with labels from 
the alphabet V x {O, I} is indeed a tree originating from a strategy profile 
of (G,va), and the second one checks for a tree originating from a strategy 
profile (o;)sen of (G,va) whether (0;)ien is a subgame perfect equilibrium 
with a payoff in between the given thresholds. The first automaton is actu- 
ally a nondeterministic tree automaton with trivial acceptance (every run of 
the automaton is accepting) and has O(|V |) states. The second automaton 
has O(kd) states and 0(1) priorities where k is the number of players and 
d is the maximum number of priorities in a player's parity condition. An 
equivalent nondeterministic parity tree automaton has 2° (kdiog ka) states 
and O(kd) priorities (Muller and Schupp, 1995). Finally, we construct the 
product automaton of the first nondeterministic parity tree automaton with 
the one constructed from the alternating one. As the former automaton 
works with trivial acceptance, the construction is straightforward and leads 
to a nondeterministic parity tree automaton with ov) .20 (kdlog kd) States 
and O(kd) priorities. Obviously, the tree language defined by this automa- 
ton is nonempty if and only if (G,va) has a subgame perfect equilibrium 
with a payoff in between the given thresholds. By (Emerson et al., 1993) 
nonemptiness for nondeterministic parity tree automata can be decided in 
time polynomial in the number of states and exponential in the number of 
priorities. Q.E.D. 


The exact complexity of SPE remains an open problem. However, NP- 
hardness can be transferred from NE to SPE. Hence, it is unlikely that there 
exists a polynomial-time algorithm for SPE, in genera!. 


Theorem 4.5. SPE is NP-hard for co-Büchi games, even with the thresh- 
olds z = (1,0, ...,0) and y= (1, ...,1). 


Proo]. The proof is analogous to the proof of Theorem 4.3. Just note that 
the Nash equilibrium of (G, C1) constructed in the case that ¢ is satisfiable 
is also a subgame perfect equilibrium. Q.E.D. 
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4.3 Secure equilibria 


For secure equilibria we concentrate on two-player games as it is done in 
(Chatterjee et al., 2004a), where secure equilibria were introduced. If there 
are only two players, then there are only four possible payoffs for a secure 
equilibrium: (0,0), (1,0), (0,1), and (1,1). For each of these payoffs, we aim 
to characterise the existence of a secure equilibrium that has this payoff and 
analyse the complexity of deciding whether there exists such an equilibrium. 


Lemma 4.6. Let (G,va) be a two-player game with determined winning 
conditions. Then (G,va) has a secure equilibrium with payoff (0,0) if and 
only if no player has a winning strategy. 


Proo]. Clearly, if (0,7) is a secure equilibrium with payoff (0,0), then no 
player can have a winning strategy, since otherwise (0,7) would not even 
be a Nash equilibrium. On the other hand, assume that no player has a 
winning strategy. By determinacy, there exist a strategy o of player 1 that 
guarantees payoff 0 for player 2 and a strategy 7 of player 2 that guarantees 
payoff 1 for player 1. Hence, (0,7) is a Nash equilibrium. But it is also 
secure since every player receives the lowest possible payoff. Q.E.D. 


Theorem 4.7. The problem of deciding whether in a two-player parity 
game there exists a secure equilibriurn with payoff (0,0) is in UP nco-UP. If 
the number of priorities is bounded, the problem is decidabie in polynomial 
time. 


Proo]. By Lemma 4.6, to decide whether there exists a secure equilibrium 
with payoff (0,0), one has to decide whether neither player 1 nor player 2 
has a winning strategy. For each of the two players, existence (and hence 
also non-existence) of a winning strategy can be decided in UP n co-UP 
(Jurdziriski, 1998). By first checking whether player 1 does not have a 
winning strategy and then checking whether player 2 does not have one, we 
get a UP algorithm for the problem. Analogously, one can deduce that the 
problem is in co-UP. 

If the number of priorities is bounded, deciding the existence of a win- 
ning strategy can be done in polynomial time, so we get a polynomial-time 
algorithm for the problem. Q.E.D. 


Lemma 4.8. Let (G,va) be a two-player game. Then (G,va) has a se- 
cure equilibrium with payoff (1,0) or payoff (0,1) if and only if player 1 or 
player 2, respectively, has a strongly winning strategy. 


Proo]. We only show the claim for payoff (1,0); the proof for payoff (0,1) 
is completely analogous. Clearly, if o is a strongly winning strategy for 
player 1, then (o,7) is a secure equilibrium for any strategy 7 of player 2. 
On the other hand, if (0,7) is a secure equilibrium with payoff (1,0), then 
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for any strategy 7/ of player 2 the strategy profile (0,77 has payoff (1, 0), 
hence g is strongly winning. Q.E.D. 


Theorem 4.9 (Chatterjee et al., 2004a). The problem of deciding whether 
in a two-player parity game there exists a secure equilibrium with payoff 
(1,0), or payoff (0,1), is co-NP-complete. If the number of priorities is 
bounded, the problem is in P. 


Proo]. Ey Lemma 4.8, deciding whether a two-player parity game has a 
secure equilibrium with payoff (1,0) or (0,1) amounts to deciding whether 
player 1 respectively player 2 has a strongly winning strategy. Assume that 
the game has parity winning conditions Q; and Q3. Then player 1 or player 2 
has a strongly winning strategy if and only if she has a winning strategy 
for the condition Q; A AQ», respectively Qs A AQ), a Streett condition. The 
existence of such a strategy can be decided in co-NP (Emerson and Jutla, 
1988). Hence, the problem of deciding whether the game has a secure 
equilibrium with payoff (1,0), or (0,1), is also in co-NP. 

In (Chatterjee et al., 2007) the authors showed that deciding the exis- 
tence of a winning strategy in a two-player zero-sum game with the conjunc- 
tion of two parity conditions as its winning condition is already co-NP-hard. 
It follows that the problem of deciding whether a player has a strongly win- 
ning strategy in a two-player parity game is co-NP-hard. 

If the number of priorities is bounded, we arrive at a Streett condition 
with a bounded number of pairs, for which one can decide the existence of a 
winning strategy in polynomial time (Emerson and Jutla, 1988), so we get 
a polynomial-time algorithm. Q.E.D. 


Lemma 4.10. Let (G,va) be a two-player game with a determined pair 
(Win., Win2) of prefix-independent winning conditions. Then (G,va) has 
a secure equilibrium with payoff (1,1) if and only if there exists a play rtt 
with payoff (1, 1) such that for all k < w no player has a strongly winning 
strategy in (G,7(k)). 


Proo]. Clearly, if (o,7) is a secure equilibrium with payoff (1, 1), then 7 := 
(o,T) is a play with payoff (1,1) such that for all k < w no player has 
a strongly winning strategy in (G,a(k)), since otherwise one player could 
decrease the other players payoff while keeping her payoff at 1 by switching 
to her strongly winning strategy at vertex 7(k). 

Assume that there is a play +r with payoff (1,1) such that for all k < w 
no player has a strongly winning strategy in (G,a(k)). Ey determinacy, 
there exists a strategy gı of player | and a strategy 7: of player 2 such that 
cı and 77 are retaliating strategies in (G, 7(k)) for each k < w. Similarly to 
the proof of Lemma 4.1, we define a new strategy o of player 1 for (G, va) 
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by 


oi(hov) otherwise. 


o(hv) ee H1) if hve 2(0)...2(k) < rr, 


where in the latter case h = hl . ba, and hi is the longest prefix of h still 
being a prefix of zt. Analogously, one can define a corresponding strategy T 
of player 2 for (G,va). It follows that the strategy profile (ø, T) has payoff 
(1, 1), and for each strategy g’ of player 1 and each strategy Tı of player 2 the 
strategy profiles (o’, 7) and (ø, Ty still give payoff 1 to player 2 respectively 
player 1. Hence, (0,7) is a secure equilibrium. Q.E.D. 


Theorem 4.11 (Chatterjee et al., 2004a). The problem of deciding whether 
in a two-player parity game there exists a secure equilibrium with payoff 
(1,1) is in NP. Ifthe number of priorities is bounded, the problem is in P. 


Proo]. Ey Lemma 4.10, to decide whether there exists a secure equilibrium 
with payoff (1,1), one has to decide whether there exists a play that has 
payoff (1, 1) and remains inside the set U of vertices where no player has 
a strongly winning strategy. By determinacy, the set U equals the set of 
vertices where both players have retaliating strategies. Assume that the 
game has parity winning conditions Q; and Q2. Then a retaliating strategy 
of player 1 or player 2 corresponds to a winning strategy for the condi- 
tion Q1 V =Q respectively Qe V =Q, a Rabin condition. Since positional 
strategies suffice to win a two-player zero-sum game with a Rabin winning 
condition (Klarlund, 1992), this implies that the set U also equals the set 
of vertices where both players have positional retaliating strategies. 

An NP algorithm for deciding whether there exists a secure equilibrium 
with payoff (1,1) works as follows: First, the algorithm guesses a set X 
together with a positional strategy o of player 1 and a positional strategy 
T of player 2. Then, the algorithm checks whether o and 7 are retaliating 
strategies from each vertex v EX. If this is the case, the algorithm checks 
whether there exists a play with payoff (1, 1) remaining inside X. If so, the 
algorithm accepts, otherwise it rejects. 

The correctness of the algorithm is immediate. For the complexity, note 
that checking whether a positional strategy of player 1 or 2 is a retaliat- 
ing strategy amounts to deciding whether the other player has a winning 
strategy for the condition Q2 A =Q; respectively Qı A Qe, again a Streett 
condition, in the one-player game where the transitions of player 1 respec- 
tively player 2 have been fixed according to her positional strategy. Also, 
checking whether there exists a play with payoff (1, 1) remaining inside X 
amounts to deciding whether there exists a winning play in a one-player 
Streett game, namely the one derived from G by removing all vertices in 
X, merging the two players into one, and imposing the winning condition 
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Q1 A Q2. As the problem of deciding the existence of a winning play in a 
one-player Streett game is decidabie in polynomial time, our algorithm runs 
in (nondeterministic) polynomial time. 

If the number of priorities is bounded, we can actually compute the 
set U of vertices from where both players have a retaliating strategy in 
polynomial time, so the algorithm can be made deterministic while retaining 
a polynomial running time. Q.E.D. 
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Abstract 


The paper deals with one but widespread natural language pheno- 
menon: discourse anaphora. I make a bridge between three game- 
theoretical approaches to this phenomenon: (a) Hintikka's game- 
theoretical semantics (GTS); (b) Dekker and van Rooij's applica- 
tion of strategie games to the underspecification of anaphora; and 
(c) Abramsky's Dynamic Game Semantics. I see (b) as leading to 
a 'gamification' of a phenomenon which GTS (and other approaches 
such as Government and Binding Theory) sawasbelonging to se- 
mantics or syntax. I see (c) as solving some problems left open by 
the ‘subgame interpretation’ of GTS. The present paper draws some 
comparisons and methodological refiections prompted by the remarks 
of my commentator, Paul Dekker. 


1 Rules and language games 


One traditional view in philosophy and linguistics is that without rules 
of usage common to the speaker and the listener, communication would 
be impossible. According to it, every linguistic expression has a meaning 
which is determined by the rules for its correct use. This obviously brings 
language and games together, for it is in the latter that rules are explicitly 
given. Here are two examples of language games which iliustrate in a very 
simple and ideal way how a communication language could emerge out of 
language games. They are due to the Finnish logician Erik Stenius who 
thought that they are typical examples of the view of language advocated 
by Ludwig Wittgenstein in his later period. 

The Garden Game is played by a gardener A and his assistant B. There 
are pieces in the game, the letters ‘a’ ‘b’, 'c' , 'P’ and ‘Q', and a ftower bed 
divided into squares as in the figure below. In every square there is a plant. 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
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HH 3rdofMay 
2nd of May 


abs Ist of May 


The game amounts to this: Every day B writes on a piece of paper the 
letters ‘a’ ‘b’, 'c', and to the left of any of these letters he writes either the 
letter 'P' or the letter 'Q', according to whether the plant in the square 
corresponding for that day to the lower-case letter is in ftower or not. For 
instance, if in the rectangle for that day, the plant next to the path is in 
ftower, whereas the two others are not, B will write: 


Pa Qb Qc 


The teaching of the game is done by simple gestures of approval and 
disapproval depending on whether B writes the correct tokens on the piece 
of paper or not. 

Once the assistant masters the Garden Game, A and B move to play the 
Report Game. A does not need to accompany B any longer to the ftower- 
bed. A now partakes in the game only by receiving the written tokens from 
B. If B really follows the rules of the game, A can read off certain facts 
from what B has written. 

It is obvious that by means of the report game, A and B have created 
a small language for communication: ‘a’, 'b', and 'c' are used to denote 
certain squares, 'P' and 'Q' express certain properties, etc. These symbols 
have acquired a meaning. 

Stenius' language games had more of a philosophical purpose, namely to 
give concrete examples of Wittgensteinian language-games. They inspired 
David Lewis who formulated them in a more precise way, using notions in 
classical game theory. In doing so, Lewis thought to respond to a chal- 
lenge launched by Quine. The latter regarded with distrust conventional 
views of language and doubted that one can give a coherent account of how 
communication takes place without presupposing already some degree of 
linguistic competence. In response to Quine's challenge, Lewis formulated 
signalling games, that is, communication games played by two players, the 
Sender and the Receiver, the former sending messages or signals about the 
situation he or she is in, and the latter undertaking a certain action aftel' 
receiving it. The point to be emphasized is that the messages do not have 
a prior meaning: whatever meaning they are going to acquire, it will be the 
result of the interactive situation in the signalling game, or in other terms, 
they will be optimal solutions in the game. Let us have a closer look at the 
game-theoretical setting. 
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2 Strategie games and Nash equilibria 


Let us shortly reeall some basic notions in classical game theory. We shall 
use as an example Prisoner's dilemma: Two criminais, 1 and 2, are interro- 
gated in separate cells. If they both confess, each of them will be punished 
to stay 3 years prison. If only one of them confesses, he will be free while 
the other will be punished with 4 years in prison. If none of them confesses, 
each will stay 1 year in prison. The picture below depiets the choices and 
payoffs of the players: 


Prisoner's dilemma 


'D' stands for "don't confess" and 'C’ stands for "confess." 

As we see, a complete description of a strategic game with two players, 
requires a list of the players' action repertoires Al and A2, and a specifica- 
tion of their utility functions u; and ua- The function u; specifies, for each 
possible sequence of choices (a,b) (action profile) i's payoff. In our example 
we have: #1(D,D) = -1, ui(D,C) = -4, ue(D,C) = 0, ete. 

Given a strategic game, we are interested in optimal plays of the game, 
that is, in every player's action being the best response to the actions of his 
opponents. Consider a simple arbitrary action profile (a,b). It is a Nash 
equilibrium in the strategic game if none of the players would have been 
better off by making a different choice: 


ula, b) > wi(c, b), for any c in Al 
uzla, b) > we(a, d), for any din A2, 


Thus in the Prisonner's dilemma game, (C,C) is a Nash equilibrium, 
but there is no Nash equilibrium in the Matching pennies game. There are 
games which have two Nash equilibria, like the one below where both (L, L) 
and (R, R) are Nash equilibria. 


3 Signalling games 
3.1 Lewis' signalling games 


Stenius’ games are games of complete information in the nontechnical sense 
that when the gardener teaches his assistant the Garden game, both the 
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Gardener and the assistant know the situation each of the letters ‘a’, 'b' or 
‘c' is supposed to be associated with. They both see whether, for a particular 
day, a plant is in the corresponding square or not. Lewis's signalling games 
abandon this assumption. Their aim is to model the kind of conventions that 
associate meanings to linguistic forms. This is achieved by Lewis's signalling 
systems, that is, ideal communicative situations in which the communicator 
or Sender (S) sends a message or a signal in a particular situation, in order 
to get a Receiver (R) to undertake a course of action. 

One of Lewis's examples is that of a truck driver A trying to reverse. Ris 
assistant B, who is behind the truck, helps him to do that by making two 
kinds of gestures: If there is room to reverse, she is making the beckoning 
gesture; otherwise she is showing hands with palms outwards. The driver 
is taking a course of action conditional on his observations: if B makes the 
beckoning gestures, then he reverses; otherwise he stops. 

It is straightforward to put this game in the above format of strategic 
games. The Sender's (B's) choices are functions S : T — F, where T is a 
set of situations ("there is place" and "there is no place") and F a set of 
messazes ("beckoniny" message and "palms outwards" message), and the 
Receiver's (A's) choices are functions R : F — Act, where Act is a set of 
actions ("reverse" and "stop"). There are 4 strategies for the Sender and 4 
strategies for the Receiver. 

Each simultaneous choice by the two players gives rise to an action profile 
(S, R). Utility functions ws and u« calculate the payoffs ws(S,R) and 
ur(S,R) for each action profile (S, R). Each of them will sum up, in a 
certain way, the payoffs of each state t 


us(t, S, R), writ, S, R). 


Given that the signalling games are cooperative games, in that both 
players try to achieve a common goal, communication, we take ws(t, S, R) 
and upr(t, S, R) to be equal in such a way that their value is 1 when com- 
munication is achieved, and 0, otherwise: 


ig (BS Re te SR) 24 1 if RESO) =t 
O otherwise. 

Finally, the expected utilities ws(S,R) and wr(S, R) are certain sums of 
uit; S, R) for all situations t. For the Sender, us(S, R) is simply X-er P(t) x 
us(t, S, R), where P is the prior probability distribution over the states in 
T (which is common knowiedge). But for the Receiver things are a bit 
more complicated. Re knows only the message but not the situation the 
Sender is in when sending it. For this reason, his expected utilities will be 
conditional on equivalence classes of states. We let S, be the information 
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state the Receiver is in aftel' the Sender chooses strategy W: 
W, = {t : W(t’) = W (t)}. 
Finally we define 


ur(S,R)= X` P(t'/W:) x urlt,S, R). 


VEW: 


A strategy profile (S,R) forms a (Bayesian) Nash equilibrium ifnone ofthe 
two players would be (strictly) better off by making another decision. 

It is games of this kind which are called by Lewis signalling systems and 
which can be associated, as pointed out earlier, with linguistic meanings. 
As these games have several appropriate Nash equilibria, the choice between 
them is conventional. The interesting question is of course which one of them 
is to be chosen as the conventional one. Lewis's weil known answer is that it 
is the more salient one which is to be selected, but the question is of course 
what makes one separating Nash equilibrium more salient than another. It 
seems that Quine's challenge to give an explanatory account of conventional 
meanings without presupposing some kind of linguistic competence has not 
yet been completely met. 

Now when such a signalling system occurs recurrently in a population, 
a convention for such a problem leads to a simple language. The signals 
used acquire meaning in virtue of the fact of being associated with partic- 
ular states in the world and actions, and this association being common 
knowiedge. Ifin the truck game, the selected equilibrium is the pair (S, R), 
where 


e Sis the strategy: make the beckoning gesture, if place to reverse, and 
show the palms outwards, otherwise, 


e R is the strategy: reverse, if shown the beckoning gesture, and stop 
otherwise 


then Lewis would say that the beckoning gesture means 'Place to reverse, 
and the showing palms outwards gestures means 'No place to reverse, and 
this meaning has been settled by convention. In other words, S(t) means t. 

(Notice however, that for this to work, the game has to have separated 
equilibria: S sends different messages in different states. ) 


3.2 Parikh's interpretation games 


Parikh's games share the same format as their Lewisian relatives, with some 
variations. They are interpretation games: A Sender is sending messages 
that the Receiver tries to interpret. It is thus more appropriate to see the 
Sender's choices as functions S : T — F from a set of states to a set of 
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linguistic forms, and the Receiver's choices as functions S : F — T. There 
are two other features which distinguish Parikh's games more essentially 
from their Lewisian relatives. For one thing, messages have costs which 
enter into the determination of payoffs. For another thing, some of the 
messages possess already a meaning. The typical situation is one in which 
the Sender has three messages, I, f’ and I", with f’ and f” possessing 
already a meaning: f’ can be used only in ti (“f’ means ti") and f” means 
til. l on the other side can be used in two distinct situations: the Sender 
may send it when she is in ¢ to communicate that she is in t; and she 
may send it when she finds herself in ti, to communicate that she is in fi. 
Otherwise the setting is very much like in Lewis's signalling games. The 
Sender knows also the situation she is in, unlike the Receiver who associates 
with the state ta probability of 0.8 and with the state ¢’ a probability of 0.2. 

To give an idea of what is going on, here are two tables with the possible 
strategies of the two players 


t ti 
Sl i J i f' a 
Sender: | $2 | í i Receiver: | HI| t | |t 
S3 | f” li H2| tilt |t 
S4 f” f’ 


The fact that messages have costs leads to a different calculation of 
the payoffs of the two players than in the previous case. The expected 
utilities ws(S, R) and wp(S,R) are calculated as above, but for ws(t, S, R) = 
ur(t, S, R), the complexity of the messages matters. Let us assume that f’ 
and tf” are more complex than I. Let Compl(J) 1, and Complrj") 
Complrj'") = 2. Under the assumption that the Sender prefers to send 
cheaper messages to expensive ones, us(t,S, R) and upr(t,S, R) are now 
redefined as: 


if R(S(t)) = t 


1 
us(t, S, R) = ur(t, S, R)= { Compl(S(t)) 
0 otherwise. 


The strategie game is now depicted in the table below: 


HI H2 

SL | (0.9,0.9) | (0.1,0.1) 
S2 | (0.8,0.8) | (0.2,0.2) 
S3 | (0.4,0.4) | (0.6,0.6) 
S4 | (0.5,0.5) | (0.5,0.5) 


It can be checked that the game has only two Nash equilibria, (S1, Hı) 
and (S3, H2), but unlike Lewis, Parikh would not say that the choice be- 
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tween them is conventional, but uses the notion of Pareto optimality to 
choose between the two: 


e An action profile (S, R) is Pareto more optimal than (SI, RI) if S > Si 
and R> RI. 


The optimal solution ofthe game is thus (S/, H/)' (Cf. also van Rooij, 2002.) 


4 Signalling games and Gricean pragmatics 


In the Parikhian game we considered above, t is more likely than ti and in 
each state there are at least two forms that could express it. But because 
f is the less "complex" of the two forms, then Parikh's theory prediets 
that t will be expressed by f and ti by the more complex expression. The 
introduetion of costs leads to an ordering of messages. The game theoretical 
setting forces the Sender to consider alternative expressions he or she could 
have used together with their costs. 

Van Rooij (2002) observes that a similar conclusion is reached by Blut- 
ner's bidimensional optimality theory (OT). We do not have the space here 
to enter into details. Suffice it to say that in this theory, for the hearer to 
determine the optimal interpretation of a given form, he must also consider 
alternative expressions the speaker could have used to express that meaning 
(interpretation). And the Speaker is foteed to consider the optimal form to 
express a particular meaning. 

Blutner's bidemensional OT has been given a game-theoretical interpre- 
tation in (Dekker and van Rooij, 2000). According to it, communication 
(information exchange) is represented as a strategic game between speaker 
and hearer in the marmer described above. For a detailed comparison be- 
tween Parikhian, Lewisian and the games introduced in (Dekker and van 
Rooij, 2000), the reader is referred to (van Rooij, 2002). 

One of the interesting things in the game-theoretical setting of (Dekker 
and van Rooij, 2000) is its conneetion to Gricean Pragmatics, where the fo- 
cus is not so much on the question of how expressions acquire their meanings, 
but rather on the distinction between what is said and what is conveyed or 
implied. The former is more or less conventional, semantic meaning, while 
the latter, although not explicitly stated, is something the speaker wants 
the Hearer to understand from what is said. In a seminal paper, Paul 
Grice tried to account for such pragmatic inferences in terms of maxims 
of conversations: He thought that the Hearer is able to associate the right 
interpretation with a particular assertion on the basis of an algorithm which 
computes it out of the relevant maxims. In optimality theory, the maxims 
lead to a ranked set of constraints which allow one to select the optimal syn- 
tactic form-interpretation pair. Dekker and van Rooij (2000) observe that 
the ranking of constraints in optimality theory has the same effect as the 
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ranking of action profiles in strategic games of the sort considered above: 
The Speaker wants to communicate a certain meaning and she has to choose 
a suitable formulation for it, while the Hearer wants to associate a correct 
interpretation with that form by considering all the other alternatives the 
speaker might have used. 

Here are a couple of examples which have a bearing on the interpretation 
of anaphora. 


Example 4.1 (Dekker and van Rooij, 2000). 


John is happy. He smiles. (1) 
A girl came in. She smiles. (2) 
Bill tickled John. He squirmed. (3) 


There are two principles at work here: 


(a) The principle of salience: pronouns refer back to the (denotation of 
the) salient expression (subject expression) of the previous sentence, 
ie., ‘John’, 'a girl’ and ‘Bill’. 


(b) The naturalness principle: because of semantical facts associated with 
‘tickled’, it is natural that in (3) the pronoun refers back to John. 


In (3), principle (a) is overruled by principle (b) explaining thus why the 
correct interpretation to be associated with (3) is the one in which the head 
of the anaphorical pronoun is 'John' and not 'Bill’. 


Example 4.2 (Hendriks & de Hoop, Dekker & Van Rooij). 


Often when I talk to a doctor., the doctor{i,j} disagrees with him{i,j}' 


(4) 


There are two general principles at work here: 


(i) If two arguments of the same semantic relation are not marked as 
being identical, interpret them as being distinct. 


(ii) Don't Overlook Anaphoric Possibilities. 


Here (ii) is overruled by (i), explaining why the head of the anaphoric pro- 
noun cannot be 'the doctor’. 


I will take up in the next section an analysis of these examples in Came- 
theoretical semantics (CTS). The comparison is instructive, for CTS tries to 
account for the same phenomena in semantical terms. I will then sketch yet 
another approach to these examples in Govemment and Binding Theory. 
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5 Game-theoretical semantics (GTS) 
5.1 Semantical games for quantifiers 


Unlike communication games whose task is to model how expressions of 
the language acquire an interpretation, in semantical games associated with 
natural or formallanguages, it is presupposed that certain basic expressions 
and sentences of a language already have an interpretation. What is wanted, 
instead, is a way to give the truth-conditions of more complex sentences of 
the language by reducing them to the truth-conditions of the known ones. 
Here is a typical example from the mathematical vernacular. 

A function y = f(x) is continuous at Xa if given a number œ however 
smali, we can find € such that If(x) - f(xa)l < œ, given any x such that 
Ix-xal<c. 

The game-theoretical analysis is supposed to throw light on the inter- 
pretation of the expressions "we can find" and "given any" assuming that 
the interpretation of the other expressions (Il, i <) is already fixed (by a 
background model). This is done by a semantical game played by two play- 
ers, the existential J and respectively the universal player Y, both choosing 
individuals from the relevant universe of discourse. The choices of the first 
correspond to "we can find" and those of the second to "given any". Unlike 
strategic games, which are one shot games, semantical games have a sequen- 
tial element with later choices depending on earlier ones. Thus a play of 
the present game consists of a sequence of three choices of individuals in the 
universe: first V chooses an in a, then J chooses £, and finaliy V chooses x. 

Given the sequential nature of semantical games, it is more appropriate 
to exhibit them, not in strategic, but in extensive form 


G = (N, H, P, (ui)ien) 


where N is a coliection of players, H is a set of histories, P is a function 
attaching to each non-maximal history the player whose turn is to move, 
and w; is the utility function for player i, i.e., a function which associates 
with each maximal history in H a payoff for player i. Each maximal history 
represents a play of the game, at the end of which each of the players is 
given a payoff. The games are strictly competitive û-sum games: for each 
maximal play, one of the player is winning and the other is loosing. The 
play is a win for the existential player if the terminal formula is true (in the 
background model). Otherwise it is a win for the universal player. 

The crucial notion is that of a strategy for a player, a method which 
gives him or her the appropriate choice depending on the elements chosen 
earlier in the game. Such a strategy is codified by a mathematical function 
9 which takes as arguments the partial histories (aa, ... , an—1) in H where 
the player is to move, and gives her an appropriate choice g(aa, ... ,an-l). 


188 G. Sandu 


9 is a winning strategy if it guarantees him a win in every maximal play in 
which she uses it. 

In our particular example, a maximal play of the game is any sequence 
(a, €, X) chosen be the players in the order specified above. If the chosen 
elements stand in the appropriate relations, i.e., 


if (Ix - xol < £) then (If(x) - f(xo)1 <a), (5) 


then we deciare the play to be a win for J and a loss for Y. Otherwise, it 
is a win for V and a loss for 4. A strategy for J is any function 9 whose 
arguments are all the individuals œ chosen by V earlier in the game. 9 is a 
winning strategy if, 


if (Ix - xol < g(a)) then (If(x) - f(xo)1 <a). (6) 


Thus the continuity of a function has been characterized by the truth of the 
second-order sentence: 


gVova|(|x - xol < g(a)) > (If(x) - f(xo)1 < )| (7) 


5.2 Anaphora and the subgame interpretation 


GTS has been extensively applied to the analysis of anaphoric descriptions 
and anaphoric pronouns. For a simple illustration, consider our earlier sen- 
tence (2), reproduced here as (8): 


A girl came in. She is happy. (8) 


The semantical game associated with (8) is completely analoguous to the 
quantifier games (in fact the game involves quantifiers), except that 


e Games are divided into subgames, one for each subsentence of (8). 
e The rules of the game are extended to cover also anaphoric pronouns. 


In our example, 'She' prompts a move by the existential player who must 
now choose the unique individual available from the earlier subgames. 

The only such individual is the one introduced for "A girl", and thus 
the game-theoretical analysis correctly prediets that (8) may receive an 
interpretation in which "A girl" is the head of "She". 

There are two main problems here. 

The first one has to do with semantical games being undetermined by 
the game rules. Here are a couple of examples. 

A problem of underdetermination ariscs when there is more than one 
individual available from earlier subgames. This is the case with our earlier 
example (3) reproduced here as (9): 


Bill tickled John. He squirmed. (9) 
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In order to obtain the right interpretation of this sentence, Hintikka and 
Sandu (1991) make use oflexical rules which eneode the semantic properties 
of lexical words like "tickled" and "squirmed". Disregarding some of the 
details, such rules have the effect that only the individual denoted by "John" 
remains available for later choices. The mechanism by which Bil! is excluded 
has to do with the fact that in virtue of the semantic properties of "tickled", 
Bil! is assigned an agentive and John a patient role. The semantic properties 
of 'squirmed', on the other side, require an argument which has the patient 
role, that is, John. 

The second example concerns the correct interpretation of (4) repro- 
duced here as (10): 


Often when I talk to a doctor., the doctor{i,j} disagrees with him{i,j}. 
(10) 
Here we need additional principles which limit the sets which choices corre- 
sponding to anaphoric pronouns can be made from: the individual chosen 
as the value of an anaphoric pronoun cannot be the same individual which 
has been chosen earlier as the value of an expression in the same clause as 
the anaphoric pronoun. In the case of him{i,j} the rule has the effect that 
the chosen individual must be disjoint from the individual chosen earlier for 
"the doctor." 
We postpone the discussion of the second problem for later on. For the 
moment let us take stock. 


6 Three kinds of explanations 


There is a phenornenon of undeterrnination in sentences (3)-(4). In GTS 
it is manifest at the level of the application of the game rules to anaphoric 
pronouns: the rules do not determine completely which individual is to be 
chosen in the play of the games. For the choice to be settled, we need to 
supplement them with additional principles. But one must be clear of what 
is going on here: the additional principles do not, properly speaking, have 
any game-theoretical content. In the play of the relevant semantical game, 
it is enough for the players to lay back and wait for syntactical principles 
or thematical roles associated with lexical items to do their job. 

In Dekker and van Rooij's analysis, there is an undetermination of 
the truth-conditions of sentences (3)-(4): their semantic content ("what is 
said") must be supplemented by additional principles motivated by Grice's 
principles of rational communication (Grice's maxims of conversation). 
These principles lead to an appropriate ranking of the form-interpretation 
pairs which lend themselves to a game-theoretical interpretation in terms of 
strategie games of communication. The correct intepretation is eventually 
obtained as the solution of such a game. 
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If one looks at the two additional sets of principles in GTS and in Dekker 
and van Rooij's analysis, one notices that they are very much variations of 
each others. For instance, the principle which says that 


e the choice of (the value of) an anaphoric pronoun cannot come from 
the same set as that from which an individual chosen as the value of 
an expression being in the same clause comes from, 


is clearly the counterpart in GTS of the maxim (i) relativized to anaphoric 
pronouns (as distinguished from refiexives). 

The main difference between these two approaches lies, as I see it, in 
what is taken to belong to semantics as opposed to pragmatics. This is an 
ongoing debate. The motivation behind the GTS treatment of (3)-(4) has 
to do with a familiar conception according to which, if asemantic category 
(indexicals, refiexives, etc) have a syntactical counterpart, then, it should 
be treated semantically, even if underspecified. By relegating underspeci- 
fied anaphoric pronouns to the realm of pragmatic phenomena pertaining 
to strategic communication, Dekker and van Rooij brought this kind of un- 
determination under the incidence of strategic communication games, à la 
Parikh. The gain in game-theoretical content seems to be obvious. The 
only qualms I have is about this kind of undetermination belonging to the 
pragmatics of communication. I won't settie this matter here. I will shortly 
describe, instead, a third approach to the very same phenomena: Govern- 
ment and Binding (GB) Theory. Initially GB tries to explain the mechanism 
underlying the behaviour of pronouns in natural language in purely syntac- 
tical terms, using notions like C-command, governing category, etc. Again, 
I will be very sketchy. 

GB theory contains a class of binding principles like: 


e A refiexive pronoun must be bound in its local domain 
e A pronominal must be free in its local domain 


(Chomsky, 1986, p. 66) where the notion of "free", "bound", "Iocal domain" 
are syntactical notions matching familiar notions in logic: free variabie, 
binding, scope, etc. 

Applied to our earlier examples, these principles predict that in (4) 'him' 
and 'the doctor' cannot be coindexed. Interesting enough, these principles 
are not sufheient to yield the correct interpretation for (3). They must be 
supplemented by "thematic relations" of the same sort we used in the game- 
theoretical analysis above which, in the end, had the effect of allowing "he" 
in (3) to be coindexed with "John". (A detailed explanation is contained in 
Hintikka and Sandu, 1991.) 
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7 Semantics for dynamic game languages 


One can interpret the two preceding sections in the following way: discourse 
anaphora brought along a phenomena of underdetermination in CTS. The 
solution proposed inside the CTS community was to enrich that frame- 
work with more syntactical or lexical principles, which lacked, however, a 
game-theoretical motivation. I regard Dekker and van Rooij's analysis as 
an alternative to that move. It certainly has the merit of making a bridge 
between phenomena which traditionally were regarded as syntactical or se- 
mantical at most, and issues in strategie games of communication. 

In this section lintend to show how certain contemporary developments 
in dynamic game semantics can be seen as solving another problem (the 
second problem | mentioned above) in the CTS treatment of anaphoric 
pronouns. 

The problem | have in mind appears in any of our earlier examples, 
say (1). Inthe (Carlson and Hintikka, 1979) and (Hintikka and Kulas, 1985) 
subgame interpretation of this sentence, a subgame is played first with the 
first subsentenee of (1). If the existential player has a winning strategy in 
it, then the players move to play the second subgame, remembering the 
winning substrategy in the first subgame. For (1) this substrategy reduces 
to the choice of an individual, who is then available for being picked up as 
the value of the anaphoric pronoun in the second subgame (cf. above). 

This interpretation is problematic in at least one respect: the anaphoric 
resolution is dependent on truth, while things should go the other way 
around. The anaphoric link between the pronoun and ‘John’ or 'A girl’ is 
established only aftel' the truth of the first subsentenee has been established. 
This is manifest in the fact that what is "transmitted" from one subgame 
to another are winning strategies, and not strategies simpliciter. There is a 
general agreement, however, that the dependenee between the anaphor and 
its head in all these sentences is prior and independent of truth. This brings 
me to the purpose of this section: to present aDynamie Came Language 
and a compositional intepretation of it, due to Abramsky (2006) in which 
the strategies of a given game are determined compositionally from the 
strategies of subgames. 

| restriet my presentation only to a subfagment of Abramsky's language 
which is given by the clauses: 


p= 11At lQer lev loll, 


where | is a propositional constant, At stands for atomie first-order formu- 
las, and vy. and y||7 represent the operations of sequential and parallel 
composition, respectively. 
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Fohowing insights from Dynamic Game Logic (van Benthem, 2003), 
Abramsky represents quantifiers by 


One 2= Daw. 


This syntax is very powerful, it turns out that it can represent paraliel 
sequences of quantifiers, like Henkin's branching quantifier 


Va Jy 
Ve dw 


) A(x, V, z, w) ==> Va.dy||Vz.dw.A(z, v, z, w). 


Two interpretations are given for this language: 


(i) Statie Semantics assigns a game to each formula in such a way that 
every Iogical operator is interpreted by an operation on games. 


(ii) Dynamic semantics assigns to each game and each player associated 
with a formula, a strategy in such a way that the strategies for complex 
formulas are obtained compositionaliy from the strategies of subfor- 
mulas. 


7.1 Statie semanties (game-semanties) 


The games in the statie semantics are actualiy comparable with games in 
extensive form. They have a sequential nature, but their structure is more 
fine grained so that games associated with complex formulas are formed 
compositionaliy from simpler ones. A game has the form 


G = (M, àm} 
where M is a Concrete Data Struciure (CDS), that is, a quadruple 
M = (CM, Vm, Dm,Fm) 
such that: 
e CM is a set of celis 
e Vm is a set of values 
e DM is a set of decisions, DM C CM x VM 


e l-m is the enabling relation which determines the possible flow of 
events (decisions) 


and 


àm : CM > {3, VY} 


As pointed out, complex games are build up compositionaliy from sim- 
pier ones according to the foliowing rules (again, we restriet ourselves to 
few cases relevant for later examples). 


Games in Language 193 


Rules for atomie games: 
e Gi = ((0,0,0,0),0) 
e GAt- ((0,0,0,0),0) 


The formulation of these two games indicate the fact that there are no 
moves associated with them. 


e The rule for Gyvy. First let Gy = ((CM, VM, DM, Fm), Am) and Gy = 
((Cn. VN, DN, FN), An). Then Govy = ((CM UCN U {co}, VM U Vn U 
{I, 2}, DM U Dw U {(co, 1), (co, 2)}, Fovy ), Apv ) 


co 


Goe | Gy 


In other words, the combined game has a new position (celI), Co which 
can be filled in by the existential player with either 1 or 2. 


From now on I restriet myself to the graphical representation of the 
games. 


. Gay : 


The complex game has a new position Co which can be filled in by 
Eloise with any element in the domain. 


n . 
e Goy: 


That is, Gy is played first followed by a play of Gy. This fact is 
encoded in the enabling relation which is now defined by 


r Foye & ke, cv (r= sUAA sE Max(Gy)A Atg, c). 


which says that when a maximal play of Gy is reached, then the play 
is continued as in Gy. 


Once the games have been defined, strategies may be built up composi- 
tionally from substrategies: 
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e The strategy set in Gazy: 


We fix g, a strategy of J in Gy, and a an individual in the domain. 
We may think of o as a set which is a (partial) function. We define 
(roughly) the notion of a strategy for the existential player: 


upalo) = {(co, @)} Uo. 
That is, uPa(o) is formed by adding to the strategy g in the game y 
the new position Co filled in with the individual a. 
Finally a strategy set for the existential player is the collection of all 
strategies 


Str3(Gazy) = {up, (0): a E€ D Ao E€ Stra(G,)} 


The strategy set in Gy.y: 


We fix 7, a strategy of din Gy, and a family (Ts)scMax(c,,) Of strategies 
of din Gy, indexed by maximal histories of Gy. (We reeall that Gyy 
consists of plays of Gy followed by plays of Gy.) A strategy for the 
existential player in the game is 


a(t) t is a nonmaximal state in Gy, 
(o - (t»)s)(t) a(t) U To) (Ø) o(t) is a maximal state in Co, 
sUT,(u) t= sUu, s E Max(G,), u. is a 
state in Gy. 


In other words, 3 plays according to o in Gy, and when a maximal 
state t has been reached, she plays according to To(¢). 


Finally the strategy set for the existential player is the collection of 
all strategies: 


Str, (Gop) = 
{(o - (Ts)s) :0 E Stra(Gy) A Ys E Max(Grp)(T, E Str3(Gy))} 


7.2 Dynamic semantics: Solution concepts 


Once the games are fixed together with the strategies of the players in a 


compositional way, we can now go one step further than in GTS (extensive 


form), and bring the semantics to life, that is, define solution concepts in 


terms of the strategie interaction of the players. For this purpose, let us fix 
a concrete CDS with a strategy set Sa for each agent a E A. A strategy 
profile is defined in the usual way as a mernber of the product of all strategy 


sets: 


(Talak A € II Sæ 


acA 
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The idea is that each state ansmg from (o,)ec,4 is a maximal one 
reached by starting in the initial state O and repeatedly playing the strate- 
gies in the profile until no further moves can be made. We still need a 
way to evaluate maximal states (payoffs, utilities, truth-valuation). In the 
present case, one uses Boolean valuations functions, from the formula of the 
state into the set {0,1}. For instance, if the formula of the maximal state 
is Qa .Pa, then we let its value (in the background model) be 1 if both Qa 
and Pa hold. 


Finally, let us give an example which illustrates the treatment of anaphora. 


Example 7.1. We consider the game associated with the formula JeQa-Px 
which can be thought of as the logical form of our earlier example (2). 

The corresponding CDS has only one cell (Recall that the games for Qx 
and Px have the form (0,0,0,0),0)): 


da 


In a maximal play, the cell is filled with an element of the domain: 


da 
a 


The maximal play gets payoff 1 for Eloise if 
0a.>« 


holds in the relevant model: 
Qa and Pa 


Then we can define notions like Nash equilibria, etc. The important 
thing to notice is that whatever Nash equilibria gets selected as the solution 
of the game, the situation is going to be such that the syntactically free 
variabie "x" is going to be semantically bound by the existential quantifier. 
The dynamic game semantics has thus led to an extension of CTS in the 


direction of dynamic logic. 
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Abstract 

Gabriel Sandu tells an appealing story of naturallanguage viewed in 
tenns of games and game theory, bringing together several strands 
from the philosophical, logical, and even computationalliterature. In 
this short invited note, I will take the cruising altitude a few levels 
up from his, and show you a panoramic picture where the clamour 
of the raw facts on the ground has become just soothing, but wholly 
negligible background noise. 


1 Meaning is a many-mind notion 


What do games have to do with natural language? On the traditional view 
of linguists and logicians, syntax is about grammatical code, semantics is 
about mathematical relationships between syntactic code and structures 
in reality, while the rest of language use is the bustling but unsystematic 
world of pmgmatics. In particular, on this view, meaning does not involve 
agency of any kind: it is a 'O-agent notion’. But starting from the 1970s, 
another view emerged placing actions of language users at centre stage, 
making meaning the ‘information change’, or more general 'context change 
potential’ of linguistic expressions. Speakers or writers change the informa- 
tion states of their heaters or readers, and semantics should describe these 
changes. This action and update-oriented 'I-agent view' of meaning is the 
basis of the well-known Amsterdam paradigm of 'dynamic semantics’ devel- 
oped by Groenendijk, Stokhof and Veltman and their students, and it also 
underlies the well-known ‘discourse representation theory' of Hans Kamp 
and Irene Heim.' Of course, this move also involves shifting the agenda. In 


1 See the Handbook of Logic and Language (van Benthem and ter Meulen, 1997) for 
a survey of paradigms and sourees in dynamie semanties broadly eoneeived sinee the 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 197-209. 
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particular, it relocates the traditional boundary line between semantics and 
pragmatics in the study of language, and entire philosophical conferences 
have been devoted to that teetonic movement.é 

But once on this raad, it seems strange to stop here. Psychologists of lan- 
guage like Herb Clark (1996) have shown convincingly that much oflanguage 
use is directed toward the hearer's rather than the speaker's perspective, it 
is the hearer's uptake which determines the success of communication. And 
once you start thinking all this through, you wind up in a 'hermeneutical 
circle’ of speakers taking into account how their hearers will interpret what 
they are saying, and heaters taking into account how speakers will phrase 
what they are saying, and level aftel' level of stacked mutual information 
unfolds, leading to the iterative 'theory of mind' and mutual expectations 
that keep human behaviour stabie according to philosophers and psycholo- 
gists. It also leads naturally to game iheoru, since that is where these circles 
find their resting place in refiective and action equilibria. 


2 Games have a history with natural language 


Indeed, the idea that natural language has an intimate relationship with 
games has recurred through the 20th century. Im the 1950s, the later 
Wittgenstein famously moved away from the crystalline logical structure 
of the Tmctatus to a paradigm of rule-generating ‘language games’, and as 
Gabriel Sandu shows, authors like Stenius tried to put more substance into 
the game metaphor. Also in the 1950s, maybe under the infiuence of the 
then nascent game theory.f various proposals were made for analyzing logic 
in terms of ‘logic games', casting basic logical activities like argumenta- 
tion (Lorenzen, 1955) or model comparison (Ehrenfeucht-Fraissé; cf. Ehren- 
feucht, 1957) as two-player games, with winning strategies encoding proofs, 
modeis, or invariance relations, as the case might be." In particular, Gabriel 
Sandu discusses one of these, Hintikka's evaluation games for first-order 
logic (Hintikka, 1973), which later made its way into the study of natu- 
rallanguage under the name of 'Game-Theoretical Semantics’ (GTS). We 
will return to these games later, which mainly analyze the 'Iogical skeleton' 
of sentence construciion: connectives, quantifiers, and anaphoric referen- 
tial relationships. Thus, logic is the driver of the analysis here--and the 
expression 'game-theoretic' does not suggest any deep contacts with game 
theory.f 


1970s, which also run over into computer scicnce. 

2 Viewed in this way, natural language is no longer a descriptive medium, but rather a 
program.m.ing language for bringing about cognitive changes. 

3 Much of the modern history of logic and its interfaces remains to be written, since 
authors usually stay with the aftermath of the foundational era in the 1930s. 

4 (Van Benthem, 2007a) is an extensive survey and discussion of logic games today. 

5 But see below for some mathematical contacts between logic games and game theory. 
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Also in the same 1960s, another, logic-free, style of game-theoretic anal- 
ysis for natural language came up in Lewis' work (cf. Lewis, 1969), going 
back to (Schelling, 1960) on signaling games. In this way of looking at 
language, Nash equilibria establish stabie meanings for lexical items, the 
smallest atoms of sentence construction. While this new view long remained 
largely a small undercurrent," it has now become a major contender, with 
the authors discussed by Gabriel Sandu: (Parikh, 1991), (Dekker and van 
Rooij, 2000), (van Rooij, 2002), (Jager and van Rooij, 2007). While logic 
games are largely about winning and losing only, these modern signaling 
games involve real preferences that communicating linguistic agents have 
about matching up intended and perceived meaning, grammatical struc- 
ture,7 as weil as computational costs in doing so. Thus, they involve more 
serious connections with game theory, and at the same time, with the topo- 
logical and metric structure of human perceptual and conceptual spaces 
(cf. Gärdenfors and Warglien (2006)). This may weil be the most serious 
encounter between linguistics and game theory today,8 and there are many 
interesting questions about its conneetion to the earlier logic-game based 
approaches like GTS. Sandu is quite right in putting this link on the map 
in his piece, though much still remains to be clarified. 


3 Evaluation games, language, and interactive logic 


The basic idea of Hintikka-style evaluation games is that two players, Ver- 
ifier and Falsifier, disagree about whether a given first-order formula ¢ is 
true in a given model M, under some assignment sof objects to variables.? 
The rules of the game refiect this scenario-and they may be seen as de- 
scribing dynamic mechanisms of evaluation or investigation of facts about 
the world. With disjunctions y Vw, Verifier must choose a disjunct to 
defend (Falsifier is opposed to both), with conjunctions yA, the choice is 
Falsifier's. A negation —y triggers a mie switch, where players change roles 
in the game for p. Moreover, quantifiers let players choose an object from 
the domain: dzy lets Verifier choose a 'witness', Vy lets Falsifier choose 
a ‘challenge’, aftel’ which play continues with the game for the formula vy. 
These moves change assignments of objects to variables, because the new 


6 Lewis himself did add interesting thoughts on 'Score-Keeping in a Language Game’. 
Also, the stream of work on common knowledge in epistemic logic relates to Lewis' 
study of conventions, though there are even some earlier sourees in the social sciences. 

7 This scenario comes part.ly from linguistic Optimality Theory and its 'rule-free' 
paradigm which casts language users as optimizing syntactic and semantic analysis 
of assertions along a set of constraint-based preferences. 


o0 


Economics and cognitive science are other natural partners in this mix, as in the newly 
established interdisciplinary Bielefeld Heisenberg Center in 'Games and Cognition'. 
lt has often been fruitful---e.g., in situation theory and in dynamic semantics-to use 


o 


first-order logic, not as a literal translation medium for nat.ural language, but as a 
methodological 'test lab' for investigating basic features of actual usage. 
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value of x now becomes the chosen object d. When the game reaches an 
atomie formula, it is checked against the cutrent assignment, and Verifier 
wins if it is true, and loses otherwise. Im all, this produces a two-agent 
scenario of changing assignments which has the following basic property. 
A formula y is true at (M,s) iff Verifier has a winning stmtegy in the 
evaluation game Game(y, M, s8). 

Much can be said about this simple game. For instanee, the dynamic 
view of logical constants as moves in a game is intriguing, and so is the muiti- 
agent ‘pulling apart’ of basic logical notions into different roles for different 
players. In this setting, players’ strategies become logicalobjects in their 
own right now, expressing 'dependencies' in interactive behaviour. This 
powerful and appealing viewpoint also underlies other logic games, and its 
many repercussions are still not fully developed today, where we seem to be 
witnessing the birth pangs of an ‘interactive logic'.10 Van Benthem (2007b) 
also points out surprising connections with the early foundations of game 
theory. In particular, the law of Excluded Middle for first-order logic says 
that Verifier can always win games ofthe form yV —y. Unpacking this by 
the above rules, the law says that either Verifier or Falsifier has a winning 
strategy in the evaluation game for any formula y. This 'determinacy' 
can be proven via Zermelo's Theorem about zero-sum two-player games of 
finite depth, which in its turn also follows from Excluded Middle plus some 
logically valid game transformations.1' Thus, semantical evaluation, and 
hence also linguistic meaning in a procedural sense, meets with classical 
game theory-a conneetion elaborated in (van Benthem, 2007b). 

In particular, viewed in this way, major issues in natural language se- 
mantics meet in inLeresLing ways wiLh basic quest.ions about garnes. Here is 
one. As we said, applying logical operations in formal languages serves as a 
model for sentence construction in natural language. And the most famous 
semantic issue arising then is Frege's Principle of compositionality: which 
says that the meaning of any linguistic expression can be determined step- 
wise, in tandem with its construction out of grammatical parts. Here, too, 
games offer a fresh perspective. As we saw, logical operations correspond 
to moves in an evaluation game--but we can also state the above scenario 
differently, since it has nothing to do with the specific games involved. Dis- 
junction and conjunction are really quite general game operations, taking 
two games C, H toa choice game Cv H or C AH starting with a choice by 
one of the players. Likewise, negation forms the obvious dual game to any 
given game. Thus, issues of linguistic compositionality become questions 


10 The recent strategie EuroCoRes Project 'LogiCCC: Modeling Intelligent Interaction in 
the humanities, computational and social sciences’ is an effort to put this development 
on the map in a much more general setting. 

11 Evaluation games for other logical languages can be much more complex, involving 
infinite histories---e.g., with the modal y-calculus: cf. (Bradfield and Stirling, 2006). 
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about game algebm, and the laws satisfied by natural game operations. For 
instance, van Benthem (2003) shows how the complete game algebra un- 
derlying first-order logic is a decidabie mixture of principles from Boolean 
Algebra plus laws for a left-, though not right-distributive operation G; H of 
sequential composition of games. Thus, if we take the evaluation game per- 
spective on naturallanguage seriously as a view of multi-agent processes, we 
must understand the algebraic structure of the natural operations creating 
complex games for compound linguistic expressions out of simple ones. 


4 Imperfect information and dependence 


But logical evaluation games in CTS have further interesting features from 
realistie game theory, viz. imperfect information. Standard logic games, 
with the above evaluation games as a prime example, assume perfect in- 
formation: players can observe each move that is played, and their only 
uncertainties are about future moves yet to be played. Cabriel Sandu has 
been one of the prime movers in a generalization, however, where the perfect 
information is abandoned in the process of semantic evaluation. Quantifier 
sequences in natural language sometimes show patterns of dependence and 
independence where it seems very natural to assume that access is blocked 
to objects chosen earlier. In the 'slash notation’ of 'independence-friendly 
logic' (‘IF logic’), a sequence like 


VadyVzdu/x2 Reyzu 


represents a ‘branching quantifier’ that can be written two-dimensionally as 


Vary 
== 
© 
Rayzu 
= 
os E 


This is true iff Verifier has a winning strategy consisting of responses to 
objects chosen by Falsifier, where the choice for u. only depends on the 
object chosen for z. In this scenario, evaluation games are no longer deter- 
mined, and they may even have only mixed equilibria in random strategies, 
letting probability into the inner sanctum of logic. There is a large technical 
literature on this generalization of classical evaluation games, but its game 
content is under debate, and Hintikka has been downplaying the original 
game motivation. Indeed, IF logic has inspired a mathematical analysis as 
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generalized predicate logic by Hodges (1997), while Väänänen (2007) ex- 
tracts the abstract logic of dependenee at stake here without game modeis. 
But the jury is still out. For instance, van Benthem (2003) analyzes 
branching quantifiers in terms of a new game operation of the parallel prod- 
uct G x H of two games being played simultaneously without intermediate 
communication.P One reason why this is of interest to natural language is 
as follows. It has been claimed that IF logic is deeply non-compositional, 
a difficulty related to the absence of natural 'sub-games' in games with im- 
perfect information (Osborne and Rubinstein, 1994). But introducing par- 
allel product operations makes the underlying game algebra compositional 
again. Sandu's artiele actually discusses another recent game-theoretic take 
on IF, stemming more from the game semantics of programming languages. 
Abramsky (2006) makes connections between IF logic and fragments of lin- 
ear logic, whose parallel products do allow for intermediate communication, 
copying moves from one sub-game to another. In all then, the question of 
the complete multi-agent game algebra behind evaluation processes for nat- 
ural language seems open, although by this stage, we have a much deeper 
mathematical take on ‘language games' than that of the 1950s. 


5 Which games 'make sense' for natural language? 


Our story so far does not exhaust the varieties of games that have been, or 
can be, brought to bear on natural language. There is a throng of further 
candidates, reftecting the many levels at which language can be studied. 


5.1 Logic games 


Tor a start, there are many logic games, and some fit natural language just 
as weil as evaluation games for sentences y against models M. In much o1- 
dinary communication, there is no model at all of the described situation to 
evaluate against. What seems much more realistic then is 'consistency man- 
agement'. We take in what the speaker says, and try to integrate this into 
consistent ‘discourse representation structures' or more abstract semantic 
information states, unless the pressures on the successive updates become 
too high, and a conversational collapse takes place. But for this consis- 
tency management, a much more appropriate scenario might be logic games 
of model construciiori, which build models for sets of formulas (Hodges, 
1997; van Benthem, 2003). In the semantics of natural language, the rele- 
vant distinction is 'dynamics of evaluation' (as in systems like DPL) versus 
‘dynamics of interpretation’, viewed as constructing a model or ‘discourse 
representation’ that makes sense of the cutrent linguistic utterances.|' 


12 Van Benthem, Ghosh & Liu (2007) provide its complete game logic and algebra. 

13 Indeed, van Benthem & van Eijck (1982) already proposed that the proper take on 
Hintikka's view of natural language would be model building games associated with 
the methad of semantic tableaux rather than with semantic model checking. 
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Interestingly, from a logical point of view, model building games are 
closely related to dialogue games fOT pTOOf, As we said earlier, these were 
already introduced by Lorenzen (1955), who wanted to explain logical va- 
lidity of inferences P > C as the existence of a winning strategy in argu- 
mentation or debate for the Proponent of the conclusion C against any 
Opponent granting the premises P. This raises the issue of infereniial 
views of language and communication, which we will not pursue here. His- 
torically, through the intermediate stage of (Blass, 1992), Lorenzen dialogue 
games eventually led to game semantics for linear logic and programming 
languages in Abramsky's style. Thus, the games that Sandu tries to conneet 
with IF logic seem quite different in spirit-but a link may be made through 
‘proof-theoretic' or category-theoretic semantics (Abramsky, 2iiii7b).14 


5.2 Signaling games 


Now add the signaling games from the recent work by Parikh, van Rooij, and 
others, mentioned above. Sandu makes a simple and prima facie seamless 
connection, but I wonder about the consistency of scenarios. Signaling 
games really represent a very different scenario of language use, prior to the 
level of logic games. A logical evaluation game can only work when two 
things have already been settled: (a) the meaning of the logical operations, 
and (b) the denotations of the basic lexical items such as predicates and 
object names. But signaling games are about establishing the latter, and 
maybe even the former, connections in the first place! 

Now in standard communication scenarios, we may assume that this 
initial phase has been achieved already, so that a global or at least alocal, 
‘linguistic convention’ has arisen. In that case, we can focus on the higher 
tasks of making claims, and convincing others. But there can be cases where 
the two tasks meet, as in the creation of the right anaphoric links, which do 
not have fixed conventional meanings. It is here where Sandu focuses his 
discussion, and I have nothing to add to that. 15 Even so, it seems fair to 
say that we have no integrated technical theory of logic games and signaling 
games, and I wonder what would be a good way of combining them. Do we 
need a game algebra for natural language which allows for composition of 
heterogeneous games of quite different sorts? 

Finally, from the viewpoint ofnaturallanguage, we have not even reached 
the complete picture of what goes on in ordinary conversation. There may 
be games that fix meanings for lexical items and for truth or falsity of ex- 
pressions whose meaning is understood. But having achieved all that, the 
‘game of conversation’ only starts, since we must now convey information, 


14 This take on natural language interpretation seems closer to Categorial Grammar and 
its semantics in the lambda calculus, cf. (van Benthem, 1991; Moortgat, 1997). 

15 Other natural examples arise in the semantic secnarlos of 'bi-directional Optimality 
Theory', many of which go beyond anaphora. 
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try to persuade others, and generally, further our goals-and maybe those of 
the ot hers as weil. In this area, Dutch-style logicians have developed a broad 
family of 'dynamic-epistemic logies' for analyzing information update and 
belief revision (cf. Baltag et al. (1998); Gerbrandy (1999); van Ditmarsch 
et al. (2007); van Benthem et al. (2006)). These systems have already been 
given game-theoretic interpretations (van Benthem, 2001, 2007c), and re- 
cent twists toward rational agency include dynamic logies for preferenee 
change (cf. the dissertations of Liu (2008), Girard (2008) and Roy (2008)). 

But conversation and communication is also an arena where game theo- 
rists have entered independently, witness the earlier references in (van Rooij, 
2002), and the recent signaling games for conversation proposed in (Fein- 
berg, 2008). Again, there is an interface between logic and game theory to 
be developed here, and it has not happened yet. 


5.3 The long term: language communities 


Finally, there is one more level where games meet with natural language. 
We have talked about lexical meaning assignment, compositional semantics 
for single expressions, about checking for truth, argumentation, or infor- 
mation flow. But these are all short-term processes that run against the 
backdrop of a much larger, and potentially infinite process, viz. natural 
language use in communities with its conventions over- time. In terms of 
computer science, the former are terminating special-purpose processes for 
concrete tasks, while the latter are about the never-ending ‘operating sys- 
tem' of natural language. Here, again, signaling games are relevant, and 
they have been applied to such diverse issues as the emergence of Gricean 
norms in pragmatics (van Rooij, 2006) or of warning signais, or argumen- 
tative strategies (Rubinstein, 2000). 

In these scenarios, a significant move takes place, from single games 
to iterated games with infinite runs. Scenarios often involve thought ex- 
periments in terms of biological fitness and evolutionary stability against 
‘invaders’ deviating from equilibrium. This is still about games and natu- 
ral language, but with a very different agenda of explaining global, rather 
than local features of linguistic behaviour. And it is a far cry from logic 
games, involving rather dynamical systems theory for computing equilibria. 
Even so, it makes sense to ask for contacts aftel' all. Infinite games like 
repeated Prisoner's Dilemma are iterated game constructions out of simple 
base games, so a discrete algebra of game constructions still makes sense 
in this extended setting. Moreover, logic games are often infinite, most 
clearly in the game semantics for linear logic and associated programming 
languages. And even from a narrowly logical point of view, questions about 
stability of long-term natural reasoning practices make just as much sense 
as they do for linguistic conventions in natural language. 
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Thus, despite some conceptual and technical differences of emphasis and 
style in the literature right now, the encounter between logic and game 
theory in the arena of natural language seems far from concluded. 


5.4 Naturallanguage as a circus: a carroussel of games 


I started by saying that naturallanguage has three main aspects of syntax, 
semantics and pragmatics. By now it will be clear that 'linguistics' can 
ask questions about many levels of language use, asking for explanations of 
simple word meanings to successful discourse, and eventually the existence 
of broad norms and conventions that hold linguistic communities together. 
It also seems clear that games, whether from inside logic or directly from 
game theory, have an attractive role to play here, as an explicit way of 
bringing out the interactive multi-agent character of language use. 

But what is the total picture? I have described natural language as a 
carroussel of games, where you can walk from one activity to another, and 
line up for the associated game. Is there a unifying principle, perhaps, one 
‘super-game'? Should we find clues in mathematics, at some level of 'deep 
game algebra’, or rather in the communicative character of homo sapiens? 
I do not know, but I think that these questions are worth asking, if 'games 
and language’ is to be more than a bunch of separate clever techniques. 


6 Coda: but what about 'logic of games'? 


Many people have heard of fruitful, and even Nobel-prize winning connec- 
tions between logic and game theory-but the above story would probably 
leave them bewildered. What we have discussed in this note are game- 
theoretic models for basic linguistic and logical activities. Dut there is a 
quite different interface, too, where logic and language play their traditional 
role, viz. the description and analysis of game forms, strategies, information 
and reasoning of agents. This involves epistemic, doxastic and dynamic log- 
ics, providing analyses of notions such as rationality and its associated game 
solution procedures. In this descriptive guise, logic plays the same role to- 
ward game theory as it does toward multi-agent systems or process theories 
in computer science. Indeed, this more traditional use of logical techniques 
constitutes the main thrust of work in my own ILLC environment in Am- 
sterdam, where games serve as one rich and intuitively appealing model of 
intelligent interaction that we want to capture by logical means.l" This is 
also the sense in which computer scientists have embraced game theory as 
a richel' model for computation (Gradel, 2004), and philosophical logicians 
as a concrete model for rationality (Stalnaker, 1997). All these contacts 
can take place while logic keeps its standard semantic and proof-theoretic 


16 Cf. (van Benthem, 1991,2006); as weil as the bundle of European projeets constituting 
the recent LoglICCC team 'Logies for Interaction’. 
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face. Of course, game-theoretic ideas can reach logic in this way, and they 
do-but there is no need for logic to 'let' game theory 'under its skin', and 
recast itself as a family of games, as we have suggested in the above. 

Nevertheless, the latter more radical view, too, has its basis in the history 
of logic, and it constitutes what Abramsky (2007b) calls logic as embodying 
process rather than logic as external process description.!" Indeed, the two 
directions are related. We can use standard logical languages to describe 
games, and then go on to use games to reinterpret what these logical lan- 
guages are. The result is a wonderful circle--carroussel?-where the two 
fields spin happily together on each other's backs. I find that interactive 
view weil in line with the spirit of the present book. 
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Abstract 


Church's Problem (1957) asks for the construction of a finite-state 
procedure that transforms any input sequence œ letter by letter into 
an output sequence 8 such that the pair (œ, 8) satisfies a given spec- 
ification. Even after the solution by Biichi and Landweber in 1969 
(for specifications in monadic second-order logic over the structure 
(N,+1)), the problem has stimulated research in automata theory 
for decades, in recent years mainly in the algorithmic study of infi- 
nite games. We present a modern solution which proceeds in several 
stages (each of them of moderate difficulty) and provides additional 
insight into the structure of the synthesized finite-state transducers. 


1 Introduction 
Fifty years ago, during the "Summer Institute of Symbolic Logic" at Cornell 
University in 1957, Alonzo Church (1957) considered a problem which is 
both simply stated and fundamental. 

Imagine a scenario in which an infinite bit stream & is to be transformed, 
bit by bit, into an infinite stream 13, as indicated in the following figure. 


output L input 
13 = 11010... 


The task is to construct a finite-state proeedure for this transformation 
when we are given a "specification" of the relation between &œ and 13, This 
specification is usually presented as a formula of a logical system. In short 
words: We have to fill the box, given a description of the desired relation 
R between input &œ and output 13, The problem is a question on automatic 
program synthesis which surprisingly can be answered positively when the 
specification language is not too expressive. 

This setting for program synthesis is fundamentally different from the 
classical framework in which terminating programs for data transformations 
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are considered. For correctness of a terminating program one relates the 
data given to the program before the start of the computation to those 
produced by the program at the end of the computation. Usually the data 
are from an infinite domain like the natural numbers. In Church's Problem, 
we deal with non-terminating computations in which inputs and outputs are 
interleaved, and the aspect of infinity enters in the dimension of time. On 
the other hand, the data processed in a single step are from a finite domain 
(in our example just {O,I}). It is this shift of infinity from data to time 
that allows to avoid undecidability results as known from the verification 
(or even synthesis) of terminating programs over infinite data spaces. 

Let us look at an example. The relation R is defined by the conjunc- 
tion of three conditions on the input-output stream (a, 8). We use self- 
explanatory notation: a(t) is the fth bit of œ (t = 0,1, ...), and J” is the 
quantifier "there exist infinitely many". 


1. Vi(a(t)= 1 —> ;3(t) = 1) 

2. =% ;3(t) = BE +1) = 0 

3. Hta(t)=0 3 Ft;3(t)=- 0 
The first two conditions are satisfied easily by producing output 1 at each 
moment. But the last condition, which has the form of a fairness constraint, 
excludes this simple solution; we cannot ignore the zero bits in œ. A natural 
idea is to alternate between outputs 0 and 1 if the inputs are only O. We 
arrive at the following procedure: 

e Jor input 1 produce output 1 

e for input 0 produce 


output 1 if last output was 0 


output 0 if last output was 1 


This procedure is executable by the finite-state transducer displayed below. 
It is presented as an automaton in which each transition is labelled with 
an input bit and the cortesponding output bit. As initial state we take, for 
example, the left-hand state. 


1/1 
7 last ka 0/1 ra last NS 
output output 
IA ATA 


0/0 


1/1 
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For a more precise statement of Church's Problem, it is necessary to fix 
the format of the specifications and that of the "solutions". Let us first 
address the solutions. Among the many concepts of transformations of se- 
quences, only a very special form is admitted for Church's Problem. Two 
aspects are relevant, the requirement of a computation "bit by bit", and the 
restrietion to "finite-state" solutions. The first requirement means that the 
output bit j3(t) has to be produced without delay aftel' receipt of the input 
bit a(t). Thus j3(t) can depend only on the input bits up to time 4, i.e., on 
the input prefix a(0)... a(t). This is a sharper restrietion than that of "con- 
tinuity" (in the Cantor topology over {O, //W), which would mean that j3(t) 
depends on some finite prefix of a—possibly a(0)... a(s) with s >t. As an 
illustration, consider the transformation Tl with Ti (aœ) = a(0)a(2)a(4).... 
It is continuous but excluded as a solution for Church's Problem (since 
T;(a)(t) depends on a(2t)). A fortiori, non-continuous transformations are 
excluded, such as T defined by T>(a@) = 111... if œ has infinitely many let- 
ters 1, otherwise T(a) = 000... (note that no finite prefix of œ determines 
even the first bit of Ty(a)). 

The restrietion to "finite-state" solutions means, in Church's words, that 
the desired sequence transformation should be realizable by a "circuit". This 
is a much stronger assumption on the admissible transformations than the 
dependency of the tth output bit on the inputs bits up to time f only: 
One requires that the computation is realizable with a fixed finite memory 
(independent of t), as with the two states of memory in our example. It 
is remarkable that this restricted type of procedure actually suffices for 
solutions of Church's Problem. In this paper we work with finite-state 
transducers in Lhe Iorrnat of Mealy autornata. Forrnally, a Jvlealy automaton 
is a structure M = (S,»,T, sad, T) where 8 is the finite set of states, X 
and f are the input alphabet and output alphabet, respectively, sa the 
initial state, 6 : 8 x © — 8 the transition function and 7:8 x } of 
the output function. In a graphical presentation we label a transition from 
P to 0(p,a) by a/T(p, a). Later we shall also allow that certain transitions 
may not produce an output letter (but the empty word c instead). The 
function ô is extended to 0* : 8 x h* — 8 by setting d*(s,¢) = s and 
5*(s,wa) = 6(0*(s,w),a) for w E &*,a € X. For the input sequence a = 
a(O)a(1)..., the output sequence j3 computed by M is given by j3(t) = 
T(0*(s9,a(0)... alt- 1)), a(t)). 

Let us now make precise the specification language. We consider the 
system of monadic second-order logic (MSO) over the successor structure 
(N, +1), also called SIS (for "second-order theory of one successor" ) or "se- 
quential calculus". This case was emphasized by Church (1963) as an open 
problem, and today it is understood that "Church's Problem" refers to SIS. 
In the logical context, one identifies a bit sequence œ with a set Py, of nat- 
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ural numbers that contains the numbers ¢ with a(t) = 1. We use s,¢, ... 
as first-order variables (ranging over natural numbers, or time instances) 
and X, Y,... as second-order variables (ranging over sets of natural num- 
bers, or bit sequences). We view the latter as predicate variables and write 
X(t) rather than ¢ E X. In SIS, one has quantifiers 4,V for both kinds of 
variables. One can define s < t by saying that t belongs to each set which 
contains s + 1 and is closed under successor. Now our example specifica- 
tion takes the form of the following SIS-formula yo(X, Y) (where we write 
Ft... for Vsdt(s <t A ...)): 


WX) > YE) A -HY (t) AnY(t+1)) A (X(t) > tY (t) 


In general, we have SIS-specifications that speak about sequences œ E 
({a, 1}ml)w and 13 E ({a, 1}/m2»~- Then we consider bit veetors rather than 
single bits, and use ml-tuples X and m2-tuples Y of second-order vari- 
ables in place of X, Y in the specifications. Similarly we write Pa for the 
predicate tuple associated with œ. Church's Problem now asks: Given an 
S1S-specification (X.Y), construct a Mealy automaton M with input al- 
phabet }, = {0,1}m! and output alphabet Y = {0,I1}m? such that for each 
input sequence & E ({a, 1}m))W an output sequence 13 E ({a, 1/m2™ is pro- 
duced by M with (N,+1) = [PaPa], or provide the arisuier that such an 
automaton does noi exist. 

An alternative view to study Church's Problem is to consider arelation 
R C {a, 1}W x {O,/}w as the definition of an infinite two-person game be- 
tween players A and B who contribute the input-, respectively the output- 
bits in turn. A play of this game is the sequence of pairs (a(t),j3(t)) of 
bits supplied for tf = 0,1,... by A and B in alternation, and the play 
(a(O),j3(O)) (e(1), 13(1)) ... is won by player B iff the pair (a, 2) belongs 
to R. A Mealy automaton as presented above defines a winning strategy 
for player B in this game; so we speak of a "finite-state winning strategy". 

In 1969, Biichi and Landweber (1969) solved Church's Problem. The 
original proof involved a complicated construction. It took some time un- 
til more accessible proofs were available. The purpose of this tutorial is 
to present a construction which is made easy by a decomposition of the 
task into sirnpler modules (following Thomas, 1995; see also Thomas, 1997; 
Gradel et al., 2002). The construction also gives extra information on the 
structure of the finite-state machines that serve as solutions. 


We wil! show the Biichi-Landweber Theorem in four stages: Ina prelim- 
inary step, the SIS-specifications are converted into automata over infinite 
words ("w-automata"). Here we use, without going into details, classical re- 
sults of Biichi and McNaughton that provide such a conversion (Biichi, 1962; 
McNaughton, 1966). We wil! il!ustrate this step by an example. Then we 
transform the obtained automaton into a game between the input player A 
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and the output player B (played essentiallyon the transition graph of the 
automaton). The task is then to decide whether B has a winning strategy 
and-if so-to construct a finite-state machine executing a winning strat- 
egy. The last two stages serve to obtain such a machine. First, we define its 
state space and transition function, and secondly we fix the output function. 
Only in this last step the decision about solvability of the specification will 
be obtained. 

There is also an alternative approach to Church's Problem, developed 
by Rabin (1972) in the framework of tree automata theory. Let us briefiy 
sketch the idea. In the situation where both players A and B select bits, 
Rabin codes a strategy of player B by a labelling of the nodes of the infinite 
binary tree: The root has no label, the directions left and right represent 
the bits chosen by A, and the labels on the nodes different from the root are 
the bits chosen by B according to the considered strategy. When player A 
chooses the bits ba, ... ,bk, he defines a path to a certain node; the label b 
of this node is then the next choice of player B. Note that a node labelling 
by bits corresponds to a subset X of the tree (containing the nodes with 
label 1). Now the paths through the (X-labelled) tree capture all plays 
that are compatible with B's strategy coded by X. One can write down 
a formula X(X) in MSO-logic over the binary tree which states that the 
winning condition is satisfied byeach path; thus X(X) says that "X is a 
winning strategy". By Rabin's Tree Theorem (Rabin, 1969) one can convert 
X(X) into a Rabin tree automaton A, (for definitions see e.g., Thomas, 
1997), check whether this automaton accepts some tree, and-if this is the 
case--construct a "regular" tree accepted by A,’ This regular tree can then 
be inLerpreLed as a finite-state winning strategy Ior player B. 

In the present notes we pursue the "linear" approach in which single 
plays are the main objects of study; so we avoid here the infinite tree struc- 
ture that captures all plays for a given strategy. 


2 From logic to automata and games 
2.1 From logic to automata 


Our first step for solving Church's Problem consists of a transformation 
of a specification y(X. Y) into a semantically equivalent but "operational" 
form, namely into a deterministic automaton A, werking over w-sequences. 
This puts Church's Problem into the framework of automata theory. It 
is remarkable that we do not have any solution of Church's Problem that 
avoids this transformation at the start-e.g., by a compositional approach 
of synthesis that is guided by the structure of the formula y. 

For an ml-tuple X and an m2-tuple Y, the input alphabet of Ay is 
{O,1l}mt™, The automaton is said to be equivalent to cp(X,Y) if it ac- 
cepts precisely those w-words which define tuples (P, Q) of sets such that 
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(N,+1) H cp[P,Q]. In the game theoretic view explained above, one may 
consider the automaton as a referee who watches the play evolving between 
players A and B that consists of the two sequences a and j3 (logically speak- 
ing: of the set tuple (Pa; Qa, built up by A and B), and who decides at 
infinity (by the acceptance condition) whether B has won or not. 

An appropriate acceptance condition is the so-called Muller condition. It 
is specified by a collection F = {F;, ..., Fk} of state sets, and the automaton 
accepts an w-word y if the set of the states visited infinitely often in the 
unique infinite run on y is one of the F;. (The sets F; are called accepting 
loops; indeed, if the states in F; are visited again and again they form a 
strongly connected set ("loop") in the transition graph of the automaton. ) 

We use here two core results of the theory of w-automata due to Biichi 
(1962) and McNaughton (1966) (see e.g., (Thomas, 1997) or (Gradel et al., 
2002) for more recent expositions). They allow to translate an SIS-formula 
into an equivalent (non-deterministic) Biichi automaton, which is then 
transformed into a deterministic Muller automaton: FOT each SIS-joTmula 
pl X.Y) one can construct an equivalent Muller automaton Ag. As a draw- 
back in this result we mention that the size of A, cannot be bounded by an 
elementary function in the length n of (see, e.g., Gradel et al., 2002); in 
other words, for no k, the k-fold iteration of the function n + 2’ can serve 
as an upper bound for the size of Ay. 

Let us illustrate the theorem for our example specification above. The 
formula Vé(a(t) = 1— j3(t) = 1) is equivalent to the Muller automaton 


with accepting loop {I} only (and where * stands for an arbitrary bit). 
The formula =4tf(t) = j3(t+ 1) = 0 is expressed by the following Muller 
automaton with accepting loops {I} and {I, 2}: 


The automaton for 4’ta(t) = 0 — F”tß(t) = 0 is best explained as follows: 
There are four states, denoted by the four possible bit-pairs, with say (0,0) 
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as initial state. From each state we have, for each bit pair (b;, b2), a transi- 
tion labelled (b;, b2) to the state (b;,b2). A set F is accepting if it satisfies 
the following condition: If the first component is 0 in some state of F, then 
the second component is 0 for some (possibly different) state of F. 

It is known how to combine Muller automata for several conditions to 
a single Muller automaton for their conjunction. We do not present it 
explicitly here for our example. Rather we turn to a variant, called "finite- 
state game with Muller winning condition". This approach, introduced by 
McNaughton (1993), is motivated by the view that the two components of 
an input letter of the Muller automaton are contributed by two players A 
and B who pursue antagonistic objectives: A aims at violating the condition 
y and B at satisfying it. 


2.2 From automata to games 


We distinguish the contribution of bits (in the general case: bit vectors) 
by two players A and B by introducing two kinds of states, called A- and 
B-states. Inan A-state, the next bit is to be picked by player A, in aB-state 
by player B. We indicate A-states by boxes and B-states by circles. The 
figure below indicates how we dissolve transitions from a state in the given 
Muller automaton by introducing intermediate states and corresponding 
transitions. 


5 o l 
AI (1) 
(0) / \o A 
0 ] 0 ] 


Note that we keep every state of the Muller automaton as an A-state. For 
each A-state q and bit b, we introduce a b-labelled transition to a new state 
called (q, b), and from (q, b) for each bit ca c-labelled transition to the state p 
which was reached from q by (°) in the original automaton. For such a state 
p we call c the corresponding "output bit", denoted out(g, b,p). (If both c- 
transitions from (q, b) lead to the same state p we agree that out(q, b,p) = 0.) 
If the input alphabet is {O,l}m' and the output alphabet {O,/}m?, we 
introduce B-states (q,b) with b € {O, 1}m', and define out(gq,b, p) as a vector 
in {0, 1}m. 

The result is a "game graph". For our example specification above, we 
can obtain the following game graph from a corresponding Muller automa- 
ton (the reader should ignore for the moment the boldface notation of some 
arrows). 
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The three conditions of our example formula are indeed captured by 
this graph. The first condition requires that a bit 1 chosen by A has to 
be answered by the bit 1 chosen by B. Ir this is violated (starting from the 
initial state 1), state 6 (and hence the loop consisting of states 6 and 7) is 
entered. The second condition says that player B should not piek two zeroes 
in succession. Ir this is violated, we would reach 6 and 7 again. We thus 
exclude states 6 and 7 from the accepting loops. The third condition (on 
fairness) means that if A chooses 0 infinitely often (which happens by going 
to 4 or 5), then B has to choose 0 infinitely often (which is only possible by 
going from 4 to 3). Altogether we deciare a loop F as accepting if it does 
not contain 6 or 7 and satisfies (4 E F V5E F >3€E F). 

How should player B piek his bits to ensure that the play visits precisely 
the states of one of these loops F infinitely often? We have to fix how to 
move from states 2, 4, 5, 7. From 7 player B has to move to 6 since there 
is no other choice. The other choices can be fixed as follows: From 2 to 1, 
from 4 to 3, and from 5 to 1 (see boldface arrows). Then, depending on 
what Player A does, a play starting in 1 will visit infinitely often the states 
1 and 2, or the states 1 to 4, or the states 1, 3, 4, 5, or the states 1 to 5. 
Each of these loops is accepting. 

We see that the acceptance condition of a Muller automaton is thus 
turned into a winning condition in the associated game (Muller winning 
condition). Furthermore, we see that player B has a winning strategy by 
fixing his moves as stated above. This winning strategy can be converted 
into a Mealy automaton when we combine again each pair of two successive 
moves (by player A and then B) into a single transition. We get an automa- 
ton with the states 1 and 3 and the following transitions: From 1 via G) 
back to 1, from 1 via (5) to 3, and from 3 via (9) and via G) back to l. 
Up to names of states (and the irrelevant initial state), this is precisely the 
Mealy automaton mentioned in the Introduction. 

In the remainder of the paper, we shall give a general construction that 
starts with a finite game graph equipped with a Muller winning condition, 
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provides the decision whether player B wins, and in this case yields a finite- 
state winning strategy. 

We add some remarks on the step from automata to game graphs. First 
let us note that there is more behind this reshaping of the model than intro- 
ducing the attractive idea of a game. The game theoretic view is most useful 
for introducing a symmetry between inputs and outputs in Church's Prob- 
lem. The two players A and B represent the antagonistic aims of falsifying 
the specification (player A, supplying input) and satisfying it (player B, sup- 
plying output). It will turn out that either A or B has a winning strategy, 
an aspect which is hidden in the original formulation of Church's Problem. 

Secondly, in studying plays over a given game graph, it is useful to ignore 
the special role occupied by the initial state. Rather we shall be interested 
in plays wherever they start, and we shall determine for each state which 
player has a winning strategy for plays starting from there. 

On the other hand, we shall simplify the model in a different detail: We 
cancel the labels on the transitions. This is motivated by the fact that the 
winning condition is formulated in terms of visits of states only, regardless 
of the labels that are seen while traversing edges. When a winning strategy 
over the unlabelled game graph is constructed, it will be easy to re-introduce 
the labels and use them for a Mealy automaton as required in the original 
formulation of Church's Problem. 

In our example, a Mealy automaton with two states was sufheient to 
solve Church's Problem for the specification yo. These two states were 
already present in the game graph G,, corresponding to the Muller au- 
tomaton A,,. (We took the states 1 and 3.) Given the game graph Gy,, 
we were able Lo fix Lhe moves of player B Irorn 1 and 3 independenL of Lhe 
"play history", i.e., independent of the path on which either of these states 
was reached. In general we shall need additional memory to define the right 
choice. We shall see that a finite memory suffices; so we can work with 
winning strategies that are implementable by finite-state machines. Such a 
finite-state machine S works on the game graph G. The states of S and of 
G should not be confused. For the solution of Church's Problem (given a 
logical formula y) we have to combine the states of S with the states of G. 
We describe this in detail at the end of the next section. 


3 Infinite games and the Biichi-Landweber Theorem 


A game graph {or arena) has the form G = (Q, QA, E) where QA C Q and 
E C Q x Q is the transition relation, satisfying v« E Q : qE # 0 (1.e., 
Vadq’ : (q,q!) E E). This condition ensures that plays cannot end in a 
deadlock. (So a subset Qo of Q induces again a game graph if from each 
q E Qo there is an edge back to Qo.) We set QB := Q \ QA. Imnthis paper 
edges will always lead from QA-states to QB-states or conversely; however 
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the results do not depend on this assumption. We restriet to finite game 
gmphs ihroughoui the paper. 

A play over C from qis an infinite sequence p = qOqglq2... with qo= q 
and (qi, qi}1) E E for i > 0. We assume that player A chooses the next 
state from a state in QA, and player B from a state in QB. Note that the 
game graph is finite whereas the plays on it are infinite; thus one speaks of 
"finite-state infinite games". 

Forrnally, a game is a pair (C, W) where C (Q, QA, E) is a game 
graph and W C QW a "winning condition" for player B. Player B wins 
the play p = qOgqiq2... if p E W, otherwise player A wins p. The use of 
such "abstract" winning conditions W is pursued in descriptive set theory, 
see (Moschovakis, 1980). In our algorithmic context we have to work with 
finitely presentabie sets W. For our considerations below, we work with two 
finite presentations of winning conditions, either by a collection F C 29 of 
sets R C Q, or by a coloring c : Q— {0, ..., k} for some natural number k. 
In the special case c : Q — {0,1} we also consider the subset F = {gE Q | 
c(q)= 1} instead. 

First we introduce two winning conditions connected with a collection 
F C 22. The first is the Mulle» winning condition; it refers to the set Inf(p) 
of states visited infinitely often in a play p: 


Inf(p) := {4 E Q 15%: p(i) = q} 


Player B wins the play p if Inf(p) E F. With these conventions we speak of 
a MulieT game (C, F). 

There is also a "weak" version of this winning condition, called weak 
Muller condition (or Staiqer- WagneT condition), which refers to the visited 
states in a play ("occurrence set"): 


Occ(p) := {q4 E Q ldi p(i) = q} 


Player B wins a play p according to the weak Muller condition if Occ(p) E F. 
We speak of the weak Muller game (C, F). 

An important special case of weak Muller games is the reachability game, 
given a set F C Q of states of the game graph (Q, QA, E). The winning 
condition for player B is satisfied for a play p if some state of p belongs to 
F. We obtain an equivalent weak Muller condition if we set F= {RCQ | 
RAF FZ}. 

The next step is to introduce the concepts of strategy, winning strategy, 
and winning region. Let us look at examples first, using the following game 
graph Cl. 
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The reachability game (Cl, {3}) with "goal set" F = {3} is won by 
player B if the play starts in state 3. Otherwise player A can avoid this 
state by going from 2 to 5 and from 6 to 4. We shall say that the winning 
reqion of player A in this game is the set {I, 2,4,5,6, 7} and that of player 
B the set {3}. As a second example, consider the condition that states 2 
and 7 both have to be visited again and again. Formally, this is the Muller 
game (Cl, F) where F consists of all sets R D {2,7}. Obviously, player B 
can win from any state: From 1 he proceeds to 2 and to 7 in alternation, 
from 5 he moves to 6, and from 3 to 4. So the winning region of A is empty 
in this case, and that of B the set of all states. Note that switching between 
the moves from | to 2 and from 1 to 7 means to use memory (here only one 
bit) when executing the strategy. 

Formally, a stmtegy fOT plosier B [rom q is a function f : Q+ > Q, 
specifying for any play prefix go... gk with qo = q and qk E QB some vertex 
r E Q with (gk,T) E E (otherwise the value of f is chosen arbitrarily). 
A play p = qOql... from go = q is played according to strategy f if for 
each qi E QB we have qj+1 = f(qo... qi)’ A strategy f for player B from 
q is called winning stmtegy fOT plosier B [rom q if any play from q which 
is played according to f is won by player B. In the analogous way, one 
introduces strategies and winning strategies for player A. We say that A 
(resp. B) wins from q if A (resp. B) has a winning strategy from q. 

For a game (C, W) with C = (Q,QA, E), the winning regions of plasjers 
A and Bare the sets WA := {q E Q | A wins from q} and WB := {q E 
Q |B wins from q}. It is obvious that a state cannot belong to both WA 
and WB; so the winning regions WA, WBare disjoint. But whether these 
sets exhaust the whole game graph is a more delicate question. One calls 
a game deiermined if WA UWé = Q, i.e., from each vertex one of the two 
players has a winning strategy. Determinacy of infinite games is a central 
topic in descriptive set theory; with the axiom of choice one can construct 
games that are not determined. For the games considered in this paper 
(i.e., games defined in terms of the operators Occ and Inf), determinacy 
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is well-known. Nevertheless we state (and prove) this claim in the results 
below, since determinacy is the natural way to show that envisaged winning 
strategies are complete: In order to show that the domain D of a strategy 
covers the entire winning region of one player, one verifies that from each 
state outside D the other player has a winning strategy. 

By the solution of a game (C, W), with game graph C = (Q, QA, E) 
and a finitely presented winning condition W, we mean two tasks: 


1. to decide for each q E Q whether gE WB or q E WA 


2. and depending on q to construct a suitable winning strategy from q 
(for player B, respectively A). 


Item 2 asks for a winning strategy that has a finite presentation. Two kinds 
of strategies will be central in the sequel, the positional and the finite-state 
strategies. A strategy i : Q+ — Q is positional if the value of i(ql ... gk) 
only depends on the "current state" gk. So a positional strategy for B 
can also be presented as a function Í : Qs — Q, or-in graph theoretical 
terms-by a subset of the edge set where from QA-states all edges are kept 
but from each QB-state precisely one edge is chosen. For the definition 
of finite-state strategies, we first observe that over a finite state set Q, a 
strategy 1 : Q+ — Q can be considered as a word function. We say that 
i is a finite-state stmtegy if it is computed by a Mealy automaton. In the 
present context a Mealy automaton is of the form S = (S, Q, Q, sa, 6,7) with 
state set S, input alphabet Q, output alphabet Q, initial state sa, transition 
function 6: Sx Q —> S, and output function 7: S x QA — Q for player A 
(respectively 7: Sx QB — Q for player B). The stmtegy is computed by S 
is now defined by is(qo ... qk) = T(0*(so, qo... qk-l), qk) (where 6*(s, w) is 
the state reached by S from s via input word w, as defined as above in the 
Introduction, and 7 is chosen for the player under consideration). 

Now we can state the main theorem on weak Muller games and on 
Muller games. We include part (a) for reasons of exposition; part (b) is the 
Biichi-Landweber Theorem. 


Theorem 3.1. (a) Weak Muller games are determined, and for a weak 
Muller game (C, F), where C has n states, one can effectively determine 
the winning regions of the two players and construct, for each state q of 
C, a finite-state winning strategy from q for the respective winning player, 
using 2” memory states. 

(b) Muller games are determined, and for a Muller game (C, F), where C 
has n states, one can effectively determine the winning regions of the two 
players and construct, for each state q of C, a finite-state winning strategy 
from q for the respective winning player, using n’ .n memory states. 


Before entering the proof, we remark that part (b) gives the desired so- 
lution of Church's Problem. For this, we proceed as in the previous section, 
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i.e., we transform a given S1S-formula y to a Muller automaton M which 
is then converted to a game graph G with Muller winning condition (see 
Section 2). Note that the game graph G inherits an initial state from M. 
Using the Biichi-Landweber Theorem, one checks whether this initial state 
belongs to the winning region of player B, and in this case one obtains a 
Mealy automaton S that realizes a winning strategy from the initial state. 
The desired finite-state strategy for the original formula y is now easily 
constructed as a product automaton from M and S. 

We give the complete definitions for the reader who wants to see the 
details. For simplicity we consider the case y( X,Y) where each player picks 
single bits only. Let M be the Muller automaton obtained from (X,Y), 
say with state set Q. The game graph G derived from M has Q as the set of 
A-states and Q x {O, 1} as the set ofB-states. Denote QU(Q x {0, 1}) by Qo. 
Let S = (S, Qo, Qo, sa, ô, T) be the Mealy automaton that realizes a finite- 
state winning strategy for player B in the Muller game over G from qo (the 
initial state of the Muller automaton). We construct the Mealy automaton 
A solving the considered instanee y of Church's Problem as follows: A has 
the state set Q x S and the initial state (qO' sa). We have to specify a 
transition for each state (q, S) and input bit b, i.e., an output bit bi and a 
new state (ql, Si). For this we compute the state q* = (q,b) of the game 
graph and the associated S-state s" = 6(s,q*). The output function of S 
yields the state ql= T(S, q*) of G and the new memory state Si= 06(s*, ql). 
The output bit bi is the value out(q,b, ql) associated to the transition from 
q* = (q,) to ql (cf. Section 2.2). 

The memory of the automaton A combines the state space of the Muller 
autornaton Mand that of Lhe strategy autornaton S. ILis not yeL weil 
understood how these two aspects play together in genera!. Our example 
in Sections 1 and 2 illustrates the case that in addition to the states of M 
no additional memory is necessary. 


4 Reachability games and weak Muller games 


In this section we outline the proof of Theorem 3.1 (a). As a preparatory 
step we solve reachability games. The fundamental construction involved in 
this solution (computation of "attractor" ) later enters also in the solution 
of weak Muller games and Muller games. For this purpose, we introduce a 
second technique, called "game simulation". It allows to transform a given 
game into another one with an "easier" winning condition, namely such that 
the method as known from reachability games applies. We shall illustrate 
this approach first for weak Muller games (in this section) and then for 
Muller games (in the next section). 
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4.1 Reachability games 
Recall that a reachability game (C, F) involves the winning condition (for 
player B) that the play should reach somewhere a state from the set F. 


Theorem 4.1. A reachability game (C, F) with C = (Q, QA, E) and F < 
Q is determined, and the winning regions WA, Ws of players A and B, 
respectively, are computable, as weil as corresponding positional winning 
strategies. 


Proo]. The proof follows a natural idea, namely to compute, fori = 0,1, ..., 
the vertices from which player B can force a visit in F within i moves. We 
call this set the ith "attractor" (for B). lts computation for increasing i is 
known from the theory of finite games (and corresponds to the well-known 
analysis of AND-OR-trees). 


Attr’;(F) := 
{q4 E Q I from q player B can force a visit of F in <i moves} 
The inductive computation is obvious: 
Attr), (F) = F, 
Attri! (F) = Attr’;(F) U {q E QB 13(q,r) E E :r E Attri (F)} 
U {q E QA IV(q,r) E E: r E Attrh(F)} 


So for step i +1 we include a state of QB if from it some edge can be chosen 
into Attri (F). We can fix such a choice for each QB-state in Attr4{'(F) 
(i = 0,1, ...) in order to build up a positional strategy. We include a state 
in QA in Attri} (F) if all edges from it lead to Attri,(F). The sequence 
Attr?, (F) C Attrh (F) C Attr?, (F) C ... becomes stationary for some index 
«< IQI. We define AttrB(F) = Ul), Attr’, (F). 

Later we shal! also use the set Attr, (F), defined in the analogous way 
for player A. 

With the inductive construction it was explained that Attrj, (F) C WB; 
furthermore we have defined a uniform positional winning strategy which 
can be applied to any state in Ws regardless of the start of the play. (For 
states in QB NF the choice of the next state is arbitrary.) 

For the converse inclusion Wg C AttrB(F) we show that AttrB(F) 
exhausts the winning region Ws. For this, we show that from each state 
in the complement of AttrB(F), player A has a winning strategy (which is 
again positional). It suffices to verify that from any state qin Q\ AttrB (F) 
player A can force to stay outside Attrj, (F) also in the next step. This is 
checked by a case distinction: If q E QA, there must be an edge back into 
Q \ Attrj, (F), otherwise all edges from q would go to Attrj, (F) whence q 
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would belong to Attrs (F). If q E QB, all edges from q must lead to Q\ 
AttrB (F), because otherwise there would be an edge to Attrj, (F) and q 
would again belong to AttrB(F). Q.E.D. 


4.2 Weak Muller games 


Ina weak Muller game (C, F), player B wins the play p iff Occ(p) E F, i.e., 
the states visited during p form a set in F. It is a useful exercise to verify 
that weak Muller games are precisely those where the winning condition 
can be expressed as a Boolean combination of reachability conditions. 

Positional strategies do not suffice in general to win weak Muller games. 
As an example, consider the following game graph and the weak Muller 
condition given by F = {{I, 2,3}} (requiring that player B should visit all 
states in order to win). 


i= | 


From vertex 2 there is no positional winning strategy: Neither the choice 
to move to I nor the choice to move to 3 will enable us to reach each vertex. 
On the other hand, a one-bit memory will do: When coming back to 2 we 
should know whether I or 3 was visited before, and then we should move 
to 3, respectively 1. A general principle derivable from this solution is to 
"rernember where we have been already". This principle corresponds to a 
simple experience of every-day life: When there is a task ahead consisting 
of several items, keep a list of what was done already (and thus of what still 
has to be done). 

We shall see that this idea suffices completely for setting up the transition 
structure of a finite-state winning strategy. Given a weak Muller game 
(C,F) with C = (Q,QA, E) and F = {F;, ..., Fk}, F; C Q, we define 
the transition structure of a Mealy automaton S with the power set of Q 
as its set of states and Q as its input alphabet. Having read the input 
word ql ... qk, its state will be {ql,"" qk}. So the initial state is 0 and 
the transition function 6 : 22 x Q — 29 is defined by 6(R,p) = Ru {p}. 
This memory of subsets of Q with the mentioned update rule is called 
appeamnce record. We shall see that this memory structure suffices for 
winning strategies in arbitrary weak Muller games over C. What remains 
is to fix the output function. For this purpose we study an expanded game 
into which the memory contents from 29 are incorporated. It will turn 
out that-based on this extra information-the winning condition can be 
reformulated for the expanded game. We call this transformation of the 
game a "game simulation". For the new game we shall provide positional 
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winning strategies, which will supply the desired output function for the 
strategy automaton S. 


4.3 Game simulation 


During a play p, the set of visited states increases weakly monotonically 
and finally reaches the value Occ(p) on which it stays fixed. Similarly the 
cardinality of the set of visited states increases until it reaches the value 
IOcc(p)l. This observation enables us to reformulate the weak Muller win- 
ning condition "Occ(p) EP'. We associate a number c(R) with each subset 
R of Q, also called its color, which conveys two pieces of information: the 
size of R, and whether R belongs to F or not. Inthe first case, we take the 
even color 2- IRI, otherwise the odd color 2- IRI- 1. Let 


2 -IRI if RE F 
c(R):= {2. IRI- 1 for Rg F 


for R £0 and set c(0) := 0. Then the following claim is obvious: 


Remark 4.2. Let p be a play and Ra, RL, R2,... be the value sequence of 
the associated appearance records. Then Occ(p) E F iff the maximal color 
in the sequence c(Ro)c( Ry )c( Re)... is even. 


This remark motivates a new winning condition over game graphs G = 
(Q, QA, E) that are equipped with a coloring c: Q— {O,...,k}. The weak 
paTity condition with respect to coloring c says: Player B wins the play 
p = TOTIT2 ... iff the maximum color in the sequence c(ro)c(r1)e(r2) -.. is 
even. Given a game graph G and a coloring c with the weak parity winning 
condition, we speak of the weak paTity game (G, c). 

Using the idea above, one transforms a weak Muller game (G, F) into 
a weak parity game (Gc): Given G = (Q,QA, E) let G’ = (2 x Q,2Q x 
QA,E'; where ((P,p), (R,T)) € E’ iff (p,T) E E and R = PU {p}, and for 
nonempty R define c(R, T) := 2- IRI if R E F, otherwise 2- IRI- 1 (and let 
c(0, T) = 0). 

Each play p = TOTI ... in Ginduces the play p’ = (0, TOX {Ta}, r1)... 
in G', which is built up according to the definition of E’, We have by 
construction that p satisfies the weak Muller condition w.r.t. F iff p' satisfies 
the weak parity condition w.r.t. c. 

This transformation of (G, F) into (G', c) (with a change of the winning 
condition) is a "game simulation". In general, we say that the game (G, W) 
with G = (Q,QA, E) is simulated by (G’, WI) with G’ = (QI, Q’,, E’) if 


there is a finite automaton S = (S, Q, sa, ò) without final states such that 


e Ql= Sx Q,Q4,=- Sx QA 


Church's Problem 227 


e ((r,p), (s, q)) E El iff (p,q) E E and o(r,p) = s (which means that a 
play p = qOql ... in C induces the play pi = (sa, go)(5(so, go),qi) ... 
over Cl), 


e a play pover C belongs to W iff the cortesponding play pi over CI 
belongs to W'. 


Ifthese conditions hold we write (C, W) <s (Cl, WI). 

This relation has an interesting consequence when the latter game al- 
lows positional winning strategies. Namely, positional strategies over C/ 
are easily translated into finite-state strategies over C: The automaton S$ 
used for the simulation realizes such a strategy when equipped with an 
output function that is obtained from the positional strategy over C/ = 


(Sx Q, Sx QA, BE’). 


Remark 4.3. Let S = (S, Q,sa,ô) and assume (C, W) <s (CI, w)- yf 
there is a positional winning strategy for player B in (CI, WI) from (sa, q), 
then player B has a finite-state winning strategy from qin (C, W). The 
analogous claim holds for player A. 


Proo]. Consider the case of player B. We extend the automaton S by an 
output function that is extracted from the winning strategy o : Qk > QI. 
It suffices to define 7 : S x QB — Q by 7s, q) := second component of 
o(s, q). Then any play p according to the strategy S belongs to W iff the 
corresponding play pi (obtained as defined via S) belongs to Wy. Since ø 
was assumed to be a winning strategy, so is the strategy executed by S. 
The case of player A is handled analogously. Q.E.D. 


We apply this remark for the concrete simulation of weak Muller games 
by weak parity games mentioned above. We show "positional determinacy" 
for weak parity games and thus-by the preceding remark-finish the proof 
of part (a) of Theorem 3.1, concerning weak Muller games. 


Theorem 4.4. A weak parity game (C, c) is determined, and one can com- 
pute the winning regions WA, WBand also construct corresponding posi- 
tional winning strategies for the players A and B. 


It may be noted that we suppressed the initial states q when speaking 
about positional winning strategies. In the proof we shall see that-as for 
reachability games-the strategies can be defined independently of the start 
state (as long as it belongs to the winning region of the respective player). 


Proo]. Let C = (Q, Qa, E) be a game graph (we do not refer to the special 
graph CI above), c: Q — {O,...,k} a coloring (w.l.o.g. k even). Set 


ei- {qE Q lc(q)= i}. 
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We first compute the attractor for B of the states with maximal color, 
which is even. When player B reaches such a state the play is won whatever 
happens later. So Ax := AttrB(Ck) is a part of the winning region of 
player B. 

The remaining nodes form the set Q \ Ax; this is again a game graph. 
Note that from each state q in Q \ Ax there is at least one edge back to 
Q \ Ak, otherwise (as seen by case distinction whether q E QA or q E QB) 
q would belong to Ax = AttrB(Ck), 

In the subgame induced by Q \ Ax, we compute Ax- ; := AttrA(Ck-1\ 
Ax); from these vertices player A can reach the highest odd color k- 1 and 
guarantee to stay away from A+, in the same way as explained above for 
reachability games (see Section 4.1). 

In both sets we can single out positional winning strategies, over A, 
for B, and over A,- ; for A. In this way we continue to adjoin "slices" of 
the game graph to the winning regions of Band A in alternation. The 
next set Ag- 2 is the set of all states q E Q\ (Ag-1 U Ax) from which 
player B can force the play to C- 2\ (Ax- ; U Ax). We denote this set by 
Ate rt A) CO, \ (A,- ; UA,)). The exponent indicates the set of 
states that induces the subgame in which the attractor computation takes 
place. In order to facilitate the notation for the general case, set Qi := 
Q \ (Aye Wea uAk). 


So we compute the sets Ak, Ak- p... Aa inductively as follows: 


A, := AttrB(Ck) 
Ay. c= Attr"— (Ck-1 \ Ag) 


and fori = k- 2,...,0: 


; Attr2' (C; \ (A1 U... U Ag)) ifi even 
AYA: p 
Attr2 (C; \ (A1 U... U Ag)) ifi odd 


The positional strategies for A and Bare chosen as explained for the 
initial cases A}, Ag- ;. Now we have 


Wp U A; and Wa U A; 


i even i add 
For the correctness, one verifies by induction on j = 0,...,k: 
k k 
U A; C Wp U A; CWa 
t=k-j i=k—j 


i even i add 
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We do not give this proof in detail; it is done in analogy to the case of 
reachability games (Section 4.1). Q.E.D. 


Returning to the solution of weak Muller games, we first note that the 
claim of Theorem 3.1(a) on the memory size of a finite-state winning strat- 
egy (2 memory states over a game graph with n states) is clear from the 
game simulation using the structure of appearance record. It is remarkable 
that this method yields a finite-state winning strategy (for either player) 
where the transition structure depends solely on the underlying game graph; 
the winning condition given by the family F enters only later in the defini- 
tion of the output function. 


5 Muller games and parity games 
5.1 An example 


As a preparation of the proof of the Büchi-Landweber Theorem 3.1(b), we 
consider a game that was introduced by Dziembowski et al. (1997). The 
game is parametrized by a natural number n; we consider here the case 
n=4. 

The game graph consists of A-states 1,2,3,4 (called number-states) and 
B-states A, B, C, D (called letter-states). There is an edge from each A-state 
to each B-state and conversely. 

The winning condition for B is the following, for a play p: The number 
of letter-statee occurruiq infinitely aften in p has to coincide with the highest 
number that occurs infinitely aften among the nurnber-siaies in p. More for- 
mally we can write IInf(p)n{A, B, C, D}I = max(Inf(p)n{1, 2,3,4}). Note 
that this defines a Muller game; the family F of accepting loops contains 
each set R such that IRn {A, B, C, D}I = max(Rn {1,2,3,4}). 

It is the job of player A to choose letter-states. H, for instance, player A 
decides aftel' some time to stick just to the letters A and D (in some order) 
and not to visit Band C anymore, then player B should infinitely often 
piek state 2 and only finitely often the larger states 3 and 4. 

From a naive point ofview it is hard to imagine how player B can win this 
game. Aftel' a finite play prefix, nothing about the set Inf(p)n {A, B, C, D} 
is decided (in fact, player A has complete freedom to go for any nonempty 
subset of {A, B, C, D} ). However, a strategy has to select one number vertex 
on the basis of the cutrent finite play prefix alone. 

Nevertheless, player B wins this game from each of the states, and the 
winning strategy illustrates again that for appropriate decisions on the fu- 
ture it may be sufheient to remember relevant facts from the past. We shall 
use a refined version of the appearance record, in which not only the visited 
states, but also their order of last visits is taken into account. Inthe present 
example, it suffices to record the list of previously visited letter-states in 
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the order of their last visits-most recently visited states noted first. If the 
current (letter-) state was already visited before, then it is shifted from its 
previous position, say at place h in the list, to the front. The position h from 
which it was taken is underlined; we call it the "hit". This structure was 
introduced by McNaughton (1965) under the name "order-vector". Later 
Gurevich and Harrington suggested in their fundamental paper (Gurevich 
and Harrington, 1982) the name "latest appearance record" (LAR) under 
which the structure is known today. 

Let us study an example. Suppose player A picks successively the letter- 
states A, C, C, D, B, D, C, D, D, .... We note this sequence on the left, and 
the associated sequence of latest appearance records on the right: 


Visited letter Reached LAR 
(A) 
(CA) 
(CA) 
(DCA) 

(BDCA) 
(DBCA) 
(CDBA) 
(DCBA) 
(DCBA) 


GVCVUAVUwWOANANS 


Now assume that player A indeed sticks to the states C and D and 
repeats these two infinitely often. Then the states A and B will finally stay 
on the last two LAR-positions and not be touched anymore. Thus the hit 
value will be only 1 or 2 from some point onwards, and the maximal hit 
value visited infinitely often will be 2. Infact, if only position 1 is underlined 
from some point onwards, then only the same letter would be chosen from 
that point onwards (and not two states C and D as assumed). 

We conclude that player B should always move to the number state 
named by the cutrent hit value. In the scenario mentioned, this would 
mean to move finally only to states 1 or 2, and to 2 infinitely often. If at 
some point player A would decide to go to one state only, this state would 
be repeated at the head of the LAR and underlined; so the maximal hit 
value visited infinitely often would be 1 (and correct again). 

We leave it to the reader to show that "to move to the number given 
by the cutrent hit value" is a winning strategy of player B in the game (see 
also Remark 5.1 below). Since the required memory is finite and the update 
rule is defined in terms of the previous LAR and the current state, this is a 
finite-state winning strategy. 

The example suggests a solution of Muller games in very close analogy to 
the case of weak Muller games, using the latest appearance record in place 
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of the appearance record. We shall introduce the LAR-structure in general 
(i.e., we take it to cover all states of the game graph under consideration 
and not only a subset, such as the letter-states in our example). From each 
LAR we extract an "index". Whereas in an appearance record we referred 
to its cardinality, we use the hit value for a LAR. Then we introduce the 
"parity condition" (a variant of the weak parity condition) as new winning 
condition and apply it in a game simulation. The solution of the parity 
game that arises in this way gives a solution of the original Muller game. 


5.2 Parity games 

Given a Muller game (G,F) with G = (Q,QA,E) and Q= {I,...,n}, we 
define the transition structure of a finite-state machine S = (S, Q, sa, ô). Its 
state set is the set of_LAR's over Q, and its purpose is to realize, given a play 
prefix i, ... ik E Q*, the computation of the corresponding LAR. Formally, 
an LAR is a pair ((jl ... /-), h) where the i. are pairwise distinct states from 
Q and 0 < h< r. The initial state is sa= (( ),0) (empty list and hit 0). 
The transition function 6 : Sx Q — S realizes the update of the LAR as 
indicated in the example above: We set 6(((21 ... ir), h), i) = ((iil ... i+), 0) if 
i does not occur in (i] ... i). Otherwise, ifi = ik we cancel i from (i] ...i,) 
to obtain (jl .. .jr-1) and set o(((t1 ... ir), h), i) = (Gjl" .jr-1), k). 

An essential ingredient of a LAR ((i; ... ip), A) is the hit set {i,,....inj 
of states listed up to and including the hit position h. Consider a play pover 
Q and the associated sequence of LAR's, denoted p’, If his the maximal hit 
assumed infinitely often in p’, we may piek a position in p’ where no unlisted 
state enters any more later in the play and where only hit values < A occur 
afterwards. From that point onwards the states listed aftel' position h stay 
fixed, and thus also the hit set for the hit value h stays fixed. We call this 
set the hit set fOT the maximal hit occurruig infinitely often in pl. 


Remark 5.1. Let p be a sequence over Q and pi be the associated sequence 
of LAR's. The set Inf(p) coincides with the hit set H for the maximal hit 
h occurring infinitely often in pl. 


Prooj]. Consider the point in p from where no new states wil! occur and 
where al! visits of states that are visited only finitely often are completed. 
Aftel' a further visit of each state in Inf(p), these states wil! stay at the head 
of the LAR's (in various orders), and the hit values will be < k := Unf(p)l. 
It remains to show that the hit value in pi reaches k again and again (so 
that k is the maximal hit occurring infinitely often in pi). If the hit was 
< k from some point onwards, the state q listed on position k would not be 
visited later and thus not be in Inf(p). Q.E.D. 


Using the remark, we can reformulate the Mul!er winning condition for 
the play p: The hit set fOT the highest hit occurruiq infinitely often in pi 
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belangs to F. This allows us to extract two data from the LAR's which are 
sufheient to decide whether the play p satisfies the Muller condition: the 
hit value and the information whether the corresponding hit set belongs to 
F. We combine these two data in the definition of a coloring of the LAR's. 
Define, for h > 0, 


— ei if {is oin} E F 
c(((ir aTi h)) t 2h -1 if {14, tn} EF 


and let c(((i1 -.. ir), 0)) = 0. 

Then the Muller condition Inf(p) E F is satisfied iff the maximal color 
occurring infinitely often in C(pl(0))C(pl(1)) ... is even. This is a "parity con- 
dition" (as introduced by Mostowski (1984) and Emerson and Jutla (1991)). 
The only difference to the weak parity condition is the reference to colors 
occurring infinitely often rather than those which occur at all. 

In general, the parity condition refers to a coloring c : Q — {O,... ,k} 
of a game graph C; it is the following requirement on a play p: 


V (Si: (pli) -j AFI: epli) >j) 


j even 


The pair (C, c) with this convention for the winning condition for player B 
is called a paTity game. 

Similar to the case of weak Muller games, one can set up a game simula- 
tion of a Muller game (C, F) bya parity game (CI, c): We use the finite-state 
machine § introduced before that transforms a given play pover C into the 
corresponding sequence pi of LAR's (realized in the states visited by S), 
and we use the coloring c defined above. The game graph Cl is fixed as in 
Section 4.3 above, using the new machine S. We obtain the game simulation 
(C, F) <s (CI, c) where (CI, c) is a parity game. 


Remark 5.2. There is a variant of Sin which some ofthe states are spared. 
We cancel the initial LAR's (corresponding to hit value 0), starting (over 
states 1,...,n) with the LAR ((/. .. n),1) rather than (( ),0), and keeping 
the update rule as before. With this change, one cannot distinguish between 
first and repeated visits of states, but clearly this loss of information is 
inessential for the satisfaction of the winning condition. The number of 
states of the reduced machine is then n/.n over a graph with n states. 


One can use § as the transition structure of automata realizing winning 
strategies in the Muller game (C, F). In order to provide also the out- 
put function, we have to solve parity games, again by positional winning 
strategies. 
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Theorem 5.3. A parity game (C, c) is determined, and one can compute 
the winning regions WA, WBand also construct corresponding positional 
winning strategies for the players A and B. 


Proo]. Given C = ee QA, E) with coloring c: Q —> {0,...,k} we proceed 
by induction on IQ , the number of states of C. 

The induction start (Q is a singleton) is trivia!. In the induction step 
assume that the maximal color k is even (otherwise switch the roles of 
players A and B). Let q be a state of the highest (even) color k and define 
Aa = Attrj,({q}). As the complement of an attractor, the set Q\ Aa induces 
a subgame. The induction hypothesis ensures a partition of Q \ Aa into the 
winning regions UA, Us of the two players (with corresponding positional 
winning strategies) in this subgame. 

We now distinguish two cases: 


1. From q, player B can ensure to be in Us UAa in the next step, 


2. From q, player A can ensure to be in UA in the next step. 


Let us first verify that one of the two cases applies (which gives a kind of 
local determinacy). Assume Case 1 fails. If q E QB, then all transitions 
from q have to go to UA, otherwise we would be in Case 1. By the same 
reason, if q E QA, then some transition from q goes to UA; so Case 2 applies. 

In Case 1, one shows Wa = Us UAttrB ({q}) and WA = UA, applying the 
positional strategies of the induction hypothesis over UA, UB, the attractor 
strategy over Attrj, ({q/), and (if q E QB) the choice of the next state from q 
according to Case 1. For the first claim, note that a play in Us UAttrj, ({ q}) 
either remains in Us from some point onwards, whence Player B wins by 
induction hypothesis, or it visits (by choice of player A) the attractor Aa and 
hence q again and again, so that player B wins by seeing the highest color 
(even!) repeatedly. The second claim WA = UA is now clear by induction 
hypothesis. 

We turn to Case 2. In this case we know that q E AttrA(UA) and 
consider the set Al = AttrA(UA U {q}), clearly of cardinality > 1. So we 
can apply the induction hypothesis to the subgame induced by Q\ Al. We 
obtain a partition of this domain into winning regions VA, Vg for A and B, 
with corresponding positional winning strategies. Now it is easy to verify 
WB Vg and WA VA U Al, with positional winning strategies again 
provided by the induction hypothesis and the attractor strategy over Al. 

Finally we note that the inductive construction can be turned to a re- 
cursive procedure which produces, given C and the coloring c, the desired 
winning regions and positional strategies. Q.E.D. 


The recursive procedure appearing in this proof involves a nested call 
of the inductive hypothesis, which means that for each induction step the 
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computational effort doubles, resulting in an overall exponential runtime. It 
is known that the problem "Given a parity game (C, c) and a state q, does q 
belong to the winning region of B?" is in the complexity class NP nco-NP. 
Whether this problem is decidabie in polynomial time is one of the major 
open problems in the algorithmic theory of infinite games. 

As mentioned above, Theorem 5.3 on positional determinacy of parity 
games completes the solution of Church's Problem. The claim on the num- 
bel’ of states of a finite-state winning strategy (n! .n memory states over 
a graph with n states) is clear from Remark 5.2. As shown in (Dziem- 
bowski et al., 1997), the factorial function also supplies a Iower bound on 
the memory size of winning strategies in Muller games. 

It is worth noting that the claim on positional determinacy of parity 
games also holds for infinite game graphs (however, without a statement on 
computability of winning strategies). This "infinite version" of the theorem 
can be applied for the complementation of automata over infinite trees (see 
Thomas, 1997). 


6 Conclusion 


Let us reeall the three major steps for a solution of Church's Problem: 
First we relied on a translation from the logic SIS to Muller automata, 
which were then changed into game graphs with Muller winning condition. 
From Muller games we constructed parity games via the LAR structure; 
and finally we presented a solution of parity games. All three steps are 
nontrivial. As mentioned, the first step involves a non-elementary blow-up 
(from length of formula to size of automaton). For each of the other two 
steps, an exponential time procedure was presented; a direct construction is 
possible, however, resulting in a single exponential altogether (see Zielonka, 
1998). On the other hand, our two-step approach showed that finite-state 
winning strategies for a Muller game over a graph C can be constructed 
with a transition structure that depends on C alone, and that only for the 
output function the winning condition has to be invoked. 

Church's Problem and its solution were the starting point for a highly 
active area of research in computer science, first restricted to pure automata 
theory, but in the last 20 years with a great inftuence in algorithmic ver- 
ification and program synthesis. A problem in cutrent research is to find 
classes of infinite game graphs over which games with MSO-definable win- 
ning conditions can still be solved algorithmically. Some results (on so-called 
pushdown graphs) are mentioned in (Grädel et al., 2002). Another direction 
is to modify or to generalize the specification language in Church's Problem 
(see, e.g., Rabinovich and Thomas, 2007). Ina wider context, more general 
models of games are studied, for instanee "concurrent games" (where the 
two players move simultaneously), "timed games" (generalizing the model 
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of timed automata), stochastic games (in which random moves enter), and 
multiplayer games. 
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Abstract 


We introduce a modal language which involves the concept of depen- 
dence. We give two game-theoretic definitions for the semantics of the 
language, and one inductive, and prove the equivalence of all three. 


1 Introduction 


Is it possible that in the future currency exchange rates depend only on gov- 
ernment decisions? It is perhaps possible, but it is certainly not necessary. 
In (Väänänen, 2007) we outlined the basics of the logic of dependence. In 
this paper we take it upon ourselves to start a study of the logic of "possible 
dependence". 

Ey dependenee we mean dependence as it occurs in the foliowing con- 
texts: Dependence of 


e a move of a player in a game on previous moves 
e an attribute of a database on other attributes 

e an event in history on other events 

e a variabie of an expression on other variables 

e a choice of an agent on choices by other agents. 


We claim that there is a coherent theory of such dependence with appli- 
cations to games, logic, computer science, linguistics, economics, etc. 

There is an earlier study of the closely related concept of independence in 
the form of the independence friendly logic, by Jaakko Hintikka (1996). In 
that approach independence is tied up with quantifiers. We find dependence 
a more basic and a more tractable concept than independence. Also, we 
find that dependence (or independence) is not realiy a concept limited to 
quantifiers but a more fundamental property of individuals. Likewise, we do 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 237-254. 
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not study here dependence or independence of modal operators from each 
other. 

The basic concept of our approach to the logic of dependence is the 
dependence atom: 


(p1,---;Pn,4). (1.1) 
with the intuitive meaning that q depends only on p: ... Pn. The quantities 
p:...Pn and q can be propositions or individuals, in this paper they are 


propositions. 


Definition 1.1. The modal language of dependence has formulas of the 
form: 


1. P,q, ... proposition symbols 

2. =(P1, ...,Pn, q) meaning "q depends only on P1-- -Pn" 
3. AvB 
4. ~A 
5. OA 


The logical operations DA (i.e, ~O7A) and A AB (ie, ~A V,B), 
A — B (ie, aAVB), A B (i.e, (A — B) A (B — A)), are treated as 
abbreviations. 

The intuition is that a set of nodes of a Kripke structure satisfies the 
formula = (PI,..., Pn, q) if in these nodes the truth value of q depends only 
on the truth values of p: ...Pn. Note that this criterion really assumes, 
as emphasized in a similar context in (Hodges, 1997), a set of nodes, for 
one cannot meaningfully claim that the propositional symbols true or false 
in one single node manifest any kind of dependence. Figures 1 and 2 give 
examples of dependence and lack of it. 

We think of the sentence 


DO(=(p, q)^ A) 


as being true in a Kripke structure if every node accessible from the root 
has access to a node with A in such a way that in these nodes q depends 
only on p. A practical example of such a statement could be: 


Whatever decisions the govemments make in the next 10 yoors, it is 
possible that by the yoor 2050 the sea levels rise and whether the rise 
is over 50 cm depends only on how many countries have reduced their 
greenhouse gas emissions. 


We define now the game-theoretical semantics of our modal dependence 
language: 
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P, q 


S4 


FIGURE 1. q depends only on P in X. 


<i 


eP.g 


FIGURE 2. q does not depend only on pin Y. 


Definition 1.2. The semantic game Cl (A) is defined as follows: Positions 
are of the form (s, B, d), where s is a node of the Kripke structure, B is 
a modal formula and d is a player (lor 11). In the beginning of Csem(A), 
played at Sa, the position is (sa, A, 11). The rules of the game are: 


1. Position is (s,p, d): Player d wins if p is true in s, otherwise the 
opponent wins. 


2. Position is (s, =(Pl, ... ,Pn, q), d): Player d wins. 


3. Position is (S,4A,d): The next position is (S, A, d*), where d* is the 
opponent of d. 


4. Position is (s, A V B, d): Player d chooses C from {A, B}. The next 
position is (S,C, d). 


5. Position is (s, ©A, d): Player d chooses a node si, accessible from s. 
The next position is (si, A, d). 
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@ vi:p,q 


u 
<= J 
5 Va : P, q 


FIGURE 3. A Kripke model M. 


A strategy o of d is uniform if in any two plays where d uses ø and 
the game reaches a position (s,=(Pi, ...,Pn,q),d) in the first play and 
(Si, =(Pi,... ,Pn, q), d) in the second play, with the same subformula 
=(Pi" " ,Pn,q) of A and the same truth values of Pi, ...,Pn, the truth 
value of q is also the same. (By the "same subformula" we mean the same 
formula occurring in the same position in A.) In the extreme case of =(p), 
the truth value of P has to be the same every time the game ends in position 
(S,=(p), d) with the same =(p). 

Note that the game Gsem(A) is determined and a perfect information 
game. Thus one of the players has always a winning strategy. However, 
there is no guarantee that this winning strategy is uniform (see Section 2). 
Thus the requirement of uniformity changes the nature of the game from 
determined to non-determined. Ina sense the game loses the perfect in- 
formation characteristic as the player who counts on a dependence atom 
=(Pi, ... ,Pn, q) being true has to choose the possible worlds without look- 
ing at other parameters than Pi, ...,Pn, as far as the truth of q is con- 
cerned. Rather than putting explicit information related restrictions on the 
moves of the players, we simply follow how they play and check whether the 
moves seem to depend on parameters not allowed by the winning positions 
(s, =(Pi, ... ,Pn, q), d). Ina sense, a player is allowed to know everything all 
the time, but is not allowed to use the knowiedge. 


Definition 1.3. A is true at a node s if player 11 has a uniform winning 
strategy in the game Ggn(A) at s. 
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The sentences 


Og 


oO=(p, q) 
OO(p V -,p) 


are all true at the root of the Kripke model of Figure 3. By the definition of 
the meaning of the negation, -,A is true in a node s if and only if player I has 
a uniform winning strategy in position (s, A, 11). Ey a logical consequence 
A = B in this context we mean that the formula B is true in every Kripke 
model at every node where A is true. Respectively, A < B means that 
both A = Band B = A hold. Finally, A is called valid if it is true in every 
Kripke structure at every node. 


Example 1.4. 
1. AA(A—>B)=B 
. A> (B-A) 
. (A> (B>C))\ (A> B) SAC 
-B = -A >A —B 
AVB & BvA 
ANBSBNA 
ANASA 
. AN(BAC)&(AA^B) ^C 


© NaM K QM 


. Av (BVC) & (AVB) VC 
10. =(p, qT) = =(4,p,T) 

11. (=(p, 4) N =(q, T) = =(p, T) 
12. =(p,T) => =(p, q,T) 

13. If A is valid, then so is OA 
14. O(A = B) \oA = OB 

15. oA AOB & O(A A B) 

16. OAV OB & O(AVB) 
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2 An example of non-determinacy 
Consider the Kripke model M of Figure 3. The sentence 


S(p > q) 


is clearly true at the root s of the model, as both extensions of s have an 
extension in which pand q have the same truth value. On the other hand, 
the sentence 


A: DO(=(p) A (p= q)) 


is not true at the root for the following reason. Aftel' the move of player 
I, the node is ¢f or u. Suppose it is £ Now player 11, in order not to lose 
right away, has to commit herself to =(p) and the node with pA q. Suppose 
the game is played again but Player I decides to move to node u. Now 
player 11 has to commit herself to =(p) and the node with -'p A -'q. At this 
point we see that the strategy that Player 11 is using is not uniform, for two 
plays have reached the same dependenee atom =(p) with a different truth 
value for p. This contradiets the very definition of uniformity. However, the 
sentence 


A: -,DO(=(p) A (p= q)) 


is not true either, that is, neither does Player I have a uniform winning 
strategy in position (8,A,11). To see why this is so, let us assume I has a 
winning strategy (uniform or non-uniform) in position (8,A, 11) and derive 
a contradiction. The position 


(8, DO(=(p) A (p > q)), L) 
is actually the position 
(8,=0=9(=(p) A (p > q)), 11), 
from which the game moves automatically to position 
(8,070(=(p) ^ (p = q)), D. 


So in this position, according to the rules, Player I makes a move and chooses 
according to his strategy, say, t. We are in position 


(t, -O(=(p) A (p © q)), D 


from which the game moves automatically to position 


(t, O(=(p) ^ (p > q)), 1). 
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Now it is Player II's turn to make a choice. We let her choose the node with 
p^ q. So we are in position 


(V1,=(p) A (p © q), 11) 


which leads to the position 
(VL, ==(p) V = © 9), D. 


Player I is to move. He does not want to play a(p + q) for that would lead 
to position 
(vı =p © q), D, 
that is, 
(VLp = q, 11), 


which is a winning position for Player 11. So Player I is foteed to play ==(p), 
leading to position 


(VL ==(p), D), 


that is, 

(Vl, =(p), 11). 
But this is a winning position for Player 11, too. So again I has lost. If Player 
I moved u. instead of t, the argument would be essentially the same. So we 
may conclude that I simply does not have a winning strategy in position 
(s,A,II). The game Gsem(A) is in this case non-determined. 

We may conclude that the sentence A V —A is not true at the root of 
M. Thus the Law of Excluded Middle is not valid in this logic. Also, 
the implication A — A is not valid. How can this be understood? The 
explanation lies in our game-theoretic concept of truth. For Player 11 to 
have a uniform winning strategy in position (s, A — A,II), she has to 
count on herself or Player I having a uniform winning strategy in position 
(s, A, 11). As we have seen, the game Gẹm(A) has no Gale-Stewart Theorem 
to guarantee it being determined. We have to give up-in the context of 
dependence logic-the idea that the meaning of A — B is that if A is true 
then B is true. Rather, we should think of A — B meaning that if Player I 
does not have a uniform winning strategy in Gsem(A), then Player 11 has a 
uniform winning strategy in Gsem(B). 


3 A non-idempotency phenomenon 
Consider the Kripke model N of Figure 4 and the sentence 


B: D=(p). 


It is clear that although Player 11 trivially wins every round of the game 
Gen(B) at s, she does not have a uniform winning strategy at s, because 
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p 


=p 


FIGURE 4. A Kripke model N. 


depending on which extension of s Player I chooses, the value of P is true 
or false. On the other hand, Player I does have a uniform winning strategy, 
namely he simply plays the node with P during every round of the game. 


Let us then look at 

C: D(=(p) V=(p)). 
Now Player 11 has a uniform winning strategy: // 1 plays the node with 
p, she plays the left disjunct, and otherunse the riqht disjunct. So we have 


shown that 
D(D V D) + DD. 


4 Inductive truth definition 


There is an alternative but equivalent truth definition, similar to the in- 
ductive truth definition of Hodges (1997) for Hintikka's IF logic. The basic 
concept here is a set X of nodes satisfying a formula, rather than a single 
node. We define: 


e p is true in X if p is true in every node in X. 
e pis true in X if p is false in every node in X. 


e =(P1,'" ,Pn, q) is true in X if any two nodes in X that agree about 
pi, ....P« also agree about q. 


e —=(pi,.. ,Pn, q) is true in X if X = 0. 


e A VB is true in X if X is the union of a set where A is true and a set 
where B is true (see Figure 5). 


e A ÀB is true in X if both A and Bare. 


e ©A is true in X if A is true in some set Y such that every node in X 
has an extension in Y (see Figure 5). 


e DA is true in X if A is true in the set consisting of all extensions of 
all nodes in X (see Figure 5). 
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FIGURE 5. Truth definition. 


More formally: 


Definition 4.1. A is true in X if and only if (X, A, 11) ET, where the set 
T is defined as follows: 


(T1) (X,p,1D ET iff pis true in every node in X. 
(T2) (X,p, I)E T iff ,p is true in every node in X. 


(T3) (X, =(PI, ...,Pn,q),11) ET iff any two nodes in X that agree about 
Pl, ... ,Pn also agree about q. 


(T4) (X, =(PI, ...,»« q), I) ET iff X = 0. 
(T5) (X, ,A, d) ET iff (X, A, d*) ET 


(T6) (X, AVB, 11) ET iff X is contained in the union of a set Y and a set 
Z such that (Y, A, 11) ET and (Z, B, 11) ET. 


(T7) (X, AVB, DET iff X is contained in the intersection of a set Y and 
a set Z such that (Y, A, D ET and (Z, B, D) ET. 


(T8) (X, ©A,11) ET iff (Y, A, 11) ET for some set Y such that every node 
in X has an extension in Y. 


(T9) (X, OA, D ET iff (Y,A,D E T for the set Y consisting of all exten- 
sions of all nodes in X. 


An easy induction shows that, as shown in (Hodges, 1997): 


Lemma 4.2. 


1. (X, A, d) E Timplies (Y, A, d) E T for all Y < X. The Downward 
Closure Propertsj. 
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2. (X, AN AA, 11) E Timplies X = O. The Consistency Propertg. 


From the Downward Closure Property it follows that (T6) can be re- 
placed by 


(T6)' (X, AVB, 11) ET iff X is the union of a set Y and a set Z such that 
(Y,A, 11) ET and (Z, B, 11) ET. 


and (T7) can be replaced by 
(T7)' (X%, AVB, DET iff (X, A, D ET and (X, B, D ET. 


The way we defined the game Ggm(A) there was always an initial node 
from which the game started. We can generalize the setup up a little by 
allowing a set X of initial nodes. A strategy of a player d in Ggm(A) 
is a winning stmtegy in X if the player wins every game started from a 
position (s, A, 11), where SEX. The strategy is uniform in X if in any 
two plays Pl and P2, started from positions (xv, A, 11) and (x2, A, 11), with 
Xi, x2 EX, where d uses the strategy and the game reaches a position 
(s,=(Pl, ...,Pn,q),d), with the same =(PI, ... ,Pn,q) and the same truth 
values of Pl, ... ,Pn, the truth value of q is also the same. Thus a player has 
a uniform winning strategy (in the original sense) at s iff he or she has a 
uniform winning strategy in {s}. 


Theorem 4.3. If in the game Gsem(A) Player 11 has a uniform winning 
strategy in the set X, then (X, A, 11) E T, i.e., A is true in the set X. 


Proof. Suppose 11 has a uniform winning strategy o in Gsem(Aq) in the set 
Xa. We prove by induction on subformulas A of Aa that if P(A, d) denotes 
the set of nodes s such that position (s,A,d) is reached while Gsem(Aq) is 
being played, 11 following ø, then (r(A, d), A, d) E T. This will suffice, for 
the initial position (s, Aa, 11) can be reached for any s E Xa and so it will 
follow that Aa is true in Xa. When dealing with r(A, d) we have consider 
different occurrences of the same subformula of Aa as separate. So, e.g., 
=(p) may occur in Aa in two different places and I'(=(p), d) is computed 
separately for each of them. 

Case i: X = r(p, 11) Since ø is a winning strategy, P is true at every 
sEX. Thus (X,p, 11) ET by (TD. 

Case ii: X = r(p, I). Since g is a winning strategy, =p is true at every 
sEX. Thus (X,p, D ET by (T2). 

Case iii: X = r(=(Pl, ...,Pn,q),1I). Let us consider s,t E X that agree 
about Pl, ...,Pn. Since g is a uniform strategy, s and tagree about q. Ey 
(T3), (X, =@I1,'.. ,»«, g), 11) E T. 

Case iv: X = r(=(PIl, ...,Pn, q), D. Since o is a winning strategy of 11, 
X = 0. Ey (T4), (X, =(PI, ... ,t-: q),D ET. 
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Case v: X = f(~A, d). Note that X = f(A, d*). Ey induetion hypothesis, 
(X, A, d*) ET, and henee (X, =A, d) ET. 

Case vi: X = f(A VB, 11). Note that X C Yu Z, where Y = f(A, 11) and 
Z = f(B,II). Ey induetion hypothesis, (Y, A, 11) E Tand (Z, B, 11) E T. 
Thus (X, A V B, 11) ET by (T6). 

Case vii: X = f(A VB, D. Note that X C Y N Z, where Y = f(A, D 
and Z = f(B, D. Ey induetion hypothesis, (Y, A, D ET and (Z, B, D ET. 
Thus (X, AV B, I)E T by (T7). 

Case viii: X = f(OA, 11). For eaeh sEX there is some si reaehable from 
s that 11 ehooses aeeording to her winning strategy o in position (s, © A, 11). 
Let Y be the set of all sueh si. Note that then Y C f(A, 11). By induetion 
hypothesis, (f(A, 11),A, 11) E T. Ey (T8), (X, OA, IDET. 

Case ix: X = I'(OA,I). For eaeh SEX there may be some si reaehable 
from s that I eould ehoose in position (S,OA,I). Let Y be the set of all 
sueh possible si (i.e. Y is the set of all possible extensions of all SEX). 
Ey induetion hypothesis (Y, A, I) E T. Ey (Tg), (X, ©A,D) ET. Q.E.D. 


Corollary 4.4. If 11 has a uniform winning strategy in G,,,, (A) at s, then 
A is true in £s}. 


5 Truth strategy 


We define a new game G2(A), whieh we eall the set game as follows: Po- 
sitions are of the form (X, B, d), where X is a set of nodes, B is a modal 
dependenee formula, and d is either I or 11. The rules of the game are as 
follows: 


(SI) (X,p,II): Player 11 wins ifp is true at every node in X, otherwise I 
wins. 


(S2) (X,p,/): Player 11 wins /fp is false at every node in X, otherwise I 
wins. 


(S3) (X, =(po,...,Pn, q), 11): Player 11 wins if any two nodes in X that 
agree about Pl, ... .P« also agree about q. Otherwise I wins. 


(S4) (X, =(po,...,Pn, q), 1): Player 11 wins if X = O, otherwise I wins. 
(S5) (X, =A, d): The game eontinues from (X, A, d*). 


(S6) (X, A V B, 11): Player 11 ehooses Y and Z sueh that X < Y UZ. 
Then Player I ehooses whether the game eontinues from (Y, A, 11) or 


(Z, B, 11). 


(S7) (X, AVB, I): Player 11 ehooses Y and Z sueh that X < YNZ. Then 
Player I ehooses whether the game eontinues from (Y, A, I) or (Z, B, D. 
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(S8) (X, OA, 11): Player 11 ehooses a set Y sueh that every node in X has 
an extension in Y. The next position is (Y, A, 11). 


(S9) (X, OA, D: The next position is (Y, A, I), where Y consists of every 
extension of every node in X. 


An easy induetion shows that if Player 11 has a winning strategy in 
position (X, A, d), and Y C X, then she has in position (Y, A, d), too. From 
this faet it follows that (86) ean be replaeed by 


(S6)' (X, A V B, 11): Player 11 ehooses Y and Z sueh that X = Y UZ. 
Then Player I ehooses whether the game eontinues from (Y, A, 11) or 


(Z, B, 11). 
and (87) ean be replaeed by 


(S7)' (X, AV B,D): Player I ehooses whether the game eontinues from 
(X, A, I) or (X, B, D). 


Theorem 5.1. If (X, A, 11) E T (i.e., A is true in X), then Player 11 has a 
winning strategy in Gset(A) in position (X, A, 11). 


Proof. Suppose that (Xa, Aa, 11) E T. The strategy of 11 in Gset(Aq) is to 
play in sueh a way that if the play is in Gset(Aq) in position P = (X, A, d), 
then T(P) = (X, A, d) ET. In the beginning the position is (Xa, Aa, 11) and 
indeed Aa is true at Xq, Aftel' this we have different cases before the game 
ends: 

Case 1: P = (X, aA, d). By assumption, T(P) = (X, -~A,d) ET. By (T5) 
(X, A, d*) E T. Now the game eontinues from position pi = (t, A, d*) and 
T(P') = (X, A, d*) ET. 

Case 2: P= (X, Av B, 11). By assumption, T(P) = (X, Av B, 11) ET. By 
(T6) there are Y and Z sueh that X < YUZ, (Y,A, 11) ET and (Z, B, 11) E 
T. 80 11 plays Y and Z in Gset(Aa). Now I deeides whether the game 
eontinues from position (Y, A, 11) or from position (Z, B, 11). Whiehever the 
decision is, we have (Y, A, 11) ET and (Z, B, 11) ET. 

Case 3: P= (t AVB, D. By assumption, T(P) = (X, AV B,D ET. By 
(T7)', (X,A, D E Tand (X, B,I) E T. Now the set game eontinues from 
position (X, A, I) or from position (Y, B, I, aeeording to the deeision of l. 
Whiehever the deeision is, we have (X, A, D ET and (X, B, D ET. 

Case 4: P = (t ©A,11). By assumption, T(P) = (X,©A,11) E T. By 
(T8), (Y,A,11) E T for some set Y of nodes aeeessibie from nodes in X. 
This set Y is the ehoiee of 11 in Gset(Aa). Now the game eontinues from 
position pi = (Y, A, 11) and T(P') = (Y, A, 1) E T. 
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Case 5: P= (t, ©A,I). Ey assumption, T(P) = (X, OA,D ET. Ey (Tg), 
(Y, A, I) E T for the set Y of all nodes accessible from nodes in X. Now the 
game continues from position pi = (Y, A, D and 7(P’)= (Y,A,D E T. 


At the end of the game Gset(Aa) we have to check that 11 indeed has 
won. There are again several cases: 
Case 6: P = (X,p, 11). Since T(P) = (X,p,lI) E T, p is true at every 
te X by (TI). So 11 has won. 
Case 7: P= (X,p, D. Since T(P) = (X,p, D E T, -,p is true at every 
t e X by (T2). So 11 has won. 
Case 8: P = (X, =(PI, ... ,Pn, q),11). Let s,t € X agree about Pl,--- ,Pn. 
Since (X, =(P1, ... ,Pn, q),11) ET, we can conclude from (T3) that s and t 
agree about q. Player 11 has won. 
Case 9: P= (X, =(Pl, ... ,Pn,q),D. So T(P) = (X, =(PI, ... , Pn, q),D E 
T. Ey (T4), X = O. Player 11 has won. Q.E.D. 


6 Power strategy 


We shall describe a strategy in Gsem(A) which is based on playing Gset(A) 
in the power set of the Kripke model, hence the name power- stmtegy. The 
advantage of playing in the power set is that we can in a sense play many 
games in parallel and use this to get a uniform strategy in Gsẹm(A) (see 
Figure 6). 


Theorem 6.1. If Player 11 has a winning strategy in Gset(A) in position 
(X,A,II), then in Gsem(A), she has a uniform winning strategy in X. 


Proo]. Suppose o is a winning strategy of 11 in Gset(Ag) in position 
(Xa, Aa, 11). The strategy of 11 in Gsem(Aq) is to play so that if the play 
is in position P = (t, A, d), then 11 is in the game Gset(Ag), playing g, in 
position T(P) = (X, A, d) with t EX. In the beginning the position in 
Gsem(Aq) can be any (s,Aa,II), where s E Xa. In Gset(Ag) the initial posi- 
tion is (Xa,Aq,11). So whichever P (s,Aa,l]) the game Gsem(Aq) starts 
with, we can let T(P) = (Xq,Aa,11). Aftel' this we have different cases 
before the game ends: 

Case 1: P= (t, -A, d). Ey assumption, T(P) = (X, -,A, d) with t EX. 
Now the game continues from position pi = (t, A, d*) in GgenfAa) and from 
position T(P') = (X, A, d*) in Gset(Aa). 

Case 2: P = (t, AVB, 11). Ey assumption, T(P) = (X, AVB, 11) such that 
t EX. By (S6) the strategy g gives two sets Y and Z such that X C Yu Z, 
the game Gset(Ag) continues from (Y,A,11)or (Z, B, 11). Since t E Y UZ, 
we have either t E Y or t EZ. In the first case 11 lets C = A, U= Y and 
in the second case C = B, U= Z. Now the game Gsem(Aq) continues from 
position P’ = (t,C,11)and T(P!) = (U,C, 11). 
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FIGURE 6. Power strategy. 


Case 3: P= (t AVB, D. Ey assumption, T(P) = (X, AVB, D. Ey (S77, 
the game Gset(Ag) can continue from either (X, A, I) or (X, B, I). Now the 
game Ggem(Aa) continues from position (t, C, I), where C = A or C = B, 
according to the choice of |. In either case we let T(P') = (X, C, D. 
Case 4: P = (t,OA,ll). By assumption, T(P) = (X,OA,II). By (S8), 
the strategy g gives a set Y of nodes accessible from nodes in X and the 
game Gset(Aa) continues from (Y, A, ID. Since t E X, there is an extension 
u. of tin Y. This is the choice of 7] in Gsem(Aq). Now the game continues 
from position P’ = (u, A, II) and we define 7( P’) = (Y, A, ID. 
Case 5: P = (t,OA,/). By assumption, T(P) = (X,OA,I). By (S9), the 
game Gset(Ag) continues from position (Y, A, I) for the set Y of all nodes 
accessible from nodes in X. Since t EX, the extension u. of t chosen by J 
is bound to be in Y. Now the game continues from position pi = (u, A, l) 
and we let r(P’) = (Y,A, D. 

At the end of the game Gsem(Aq) we have to check that II indeed has 
won. There are again several cases: 
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Case 6: P = (tp, 11). Since T(P) = (X,p, 11) and øg is a winning strategy, 
pis true att So 11 has won Gsem(Ag). 
Case 7: P = (tp, D. Since T(P) = (X,p, D and ø is a winning strategy, 
-p is true at t. So 11 has won Gsem(Ao). 
Case 8: P= (t =(PI, „Pn, q), 11). Player 11 has won Gsem(Ao). 
Case 9: P = (t=(PL ,Pn,q),1). Now T(P) (X,=(PI, ... ,Pn,q),1). 
Since g is a winning strategy, X = O. On the other hand, by assumption, 
t EX. So this case simply cannot occur. 

Now that we know that this strategy is a winning strategy, we have to 
show that it is a uniform strategy. Suppose therefore that two plays 


Po, Weng Pm, Where P; = (ti, A;, d;) 
P$, ... Pegs where PI = (t, A;, di) 


end in the same formula Am = Aly which is of the form =(P1, ... ,Pn, q) 
and that the nodes t-, and Teej give pi, ....P« the same value. Let 


T(Pi) = (Xi, Ai, di) i =- 1, m 
T(P;) = (Xi, Aj, dj), i = 0, , mi 


be the corresponding positions in Gset(Ao). We show now by induction on 
i that m = mi, X; = X}, Ai = A, and di = dj. The case i = 0 is 
deal': Aa = Aj, Xo = X6 and do = dj = Il. The inductive proof is 
trivial, apart from the case Pi = (ti, A V B, d;), di = 11. Byassumption, 
T(Pi) = (Xi, A V B, 11). The strategy o has given the two sets Y and Z 
such that X C Y UZ, and the game Gset(Ao) continues from (Y, A, 11) or 
(Z, B, 11). Since t e YUZ, we have eithert € Y ort e Z. In the first case 11 
lets C = A, U = Y and in the second case C = B, U = Z. Now the game 
Gsem(Ao) continues from position Pi} = (£ C, II) and 7(Pi41) = (U,C,II). 
Respectively, PI = (t;,A V B, 11)and T(P’) = (Xi, A V B, 11). The strategy 
o (which does not depend on the elements ti and t;) has given the same two 
sets Y and Z, as above, and the game Gset(Ao) continues aftel' T(Pi) = 7(P/) 
from (Y, A, 11) or (Z, B, 11), according to whether t E Y or t EZ. So 
XI+l = Xia, Abad = Ajsi and Gay = di1- 

Thus t-, and the are in Xm and give the same value to pi, ..., Pn. Because 
g is a winning strategy of 11, the nodes t-, and ty must give the same value 
also to g. We have demonstrated the uniformity of the strategy. Q.E.D. 


7 The main result 
Putting Theorems 4.3, 5.1 and 6.1 together, we obtain: 


Theorem 7.1. Suppose A is a sentence of the modal dependence language, 
and X is a set of nodes of a Kripke structure. The foliowing are equivalent: 
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1. (X, A, 11) E T (ie., A is true in the set X). 
2. Player 11 has a uniform winning strategy in Csem(A) in the set X. 
3. Player 1l has a winning strategy in Cset(A) in X. 


Corollary 7.2. Suppose A is a sentence ofthe modal dependence language, 
and s is a node of a Kripke structure. The foliowing are equivalent: 


1. ({s},A,II) ET (e., A is true in the set {s}). 
2. Player 1l has a uniform winning strategy in Cgem(A) at s. 
3. Player 11 has a winning strategy in Cset(A) in {s}. 


The proved equivalence leads to easy proofs of the logical consequences 
and equivalences of Example 1.4. Let us consider, as an example 


[J(A > B) A oA = DB. 


Let X be a set of nodes of a Kripke model. Suppose O(A — B) and oA 
are true in X. Let X/ be the set of nodes accessible from nodes in X. Thus 
A — Band A are true in XI. Then by (T6)', XJ = Y UZ such that 4A 
is true in Y and B is true in Z Ey Lemma 4.2 and (T7), A A >A is true 
in Y. Ey Lemma 4.2, Y = O. So X/ = Zand B is true in XI. We have 
demonstrated that DB is true in X. 

The point of Theorem 7.1 is that the first game Cl with positions of 
the form (s, A, d) is non-determined and of imperfect information. The set 
game Cp» is determined and of perfect information. In an obvious sense 
the two games are equivalent. So we have been able to replace a 
non-determined game of imperfect information with a determined 
game of perfect information. The cost of this operation is that the de- 
termined game of perfect information is played on sets rather than elements. 
So in a sense there is an exponential cost. 


8 Further developments 


We can define =(PI,...,Pn,q) in terms of =(q) if we allow exponential 
growth of the formula size: =(PI, ...,Pn, q) is true in a set X if and only if 
the foliowing formula is: 


( PINUA PnA=(q))V 
(api ^A PnA=(q))V 
} 2" disjuncts. 


(mp1 ^ NPr A=(q)) 
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We can define =(p) if we add to our modal dependence language a 
Boolean disjunction A V, B with the obvious meaning that A V, B is true 
in a set iff A is true in the set or Bis, (and ~(A Vs B) is true only' 
if X = 0). In terms of the game Gsem(Ao) this means that in position 
(s, AV, B, 11) Player 11 chooses A or B, and in position (s, A Vs B, I) Player 
I wins. A uniform winning strategy of 11 is required to satisfy the extra 
condition that player 11 has to make the same move eoeru time the position 
(s, AVg B, 11) is encountered, however many times the game is played. With 
these conventions =(p) is logically equivalent to p Vg =p. 

Merlijn Sevenster (2008) has proved a normal form for modal dependence 
language and used it to show that the modal dependence language has in 
fact a translation into basic modal language, but again at exponential cost. 
He also shows that the satisfaction problem of modal dependence language 
is NEXP complete. 

The finite information logic (Parikh and Väänänen, 2005) is based on de- 
pendence formulas of the type =(A,, ...,A,,X), with the meaning that the 
value of the variabie x is chosen on the basis of the truth values of the formu- 
las Al, ...,A, only. The formulas Al, ...,A, are assumed to be quantifier 
free first order formulas (in fact they can be Ag formulas). Quantifiers are 
allowed only in a "guarded" situation such as dz(=(A,, ...,A,, X) A B) and 
Ve(=(Aj,...,An.x) —> B). This is equivalent to the existential-universal 
fragment of first order logic, but at exponential cost in the length of the 
formula. The point of this logic is that it captures the concept of social 
softsoare in the sense that people in social situations often make decisions 
on the basis of finite information about the parameters, indeed on the ba- 
sis of Lhe truth-values of sorne predicates, like "has a valid visa”, "speaks 
Dutch," etc. 

In full dependenee logic (Väänänen, 2007) first order logic is extended 
by dependence formulas =(Y/, ... ,Yn, X) with the meaning that the value of 
x depends only on the values of Y/, ..., Yn. This logic is equivalent to the 
existential second order logic, and is thus quite powerful. 

If dependence formulas are added to second order logic, again no proper 
extension results. We may thus conclude that adding dependence to a logic 
increases the expressive power in the "middle range" of first order logic, but 
not in the case of the relatively weak modal logic and the relatively strong 
second order logics. 
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"Science is the knowledge of consequences, and 
dependence of one fact upon another." 
-Thomas Hobbes 


1 It all depends, doesn't it? 


Dependence is one of these subtie concepts with so many connotations and 
usages, that any analysis of its meaning is predestined to fall short of some of 
its aspects. Inthe first paragraph of the book Dependenee Logic, Väänänen 
(2007), states: 


‘Dependence is a common phenomenon, wherever one looks: ecolog- 
ical systems, astronomy, human history, stock markets. With global 
warming, the dependence of life on earth on the actions of mankind 
has become a burning issue. But what is the logic of dependence?’ 


The book promises a systematic study of the concept, and to show that 
there is a mathematical theory of dependence. 

The paper in this volume (Väänänen, 2008) goes even further. It presents 
a logic of ‘possible dependence’, where the intended sense of 'dependence' 
is specified a bit further as follows: 


By dependence we mean dependence as it occurs in the following 
contexts: Dependence of 

e a move of a player in a game on previous moves 

e an attribute of a database on other attributes 

e an event in history on other events 

e a variabie of an expression on other variables 

e a choice of an agent on choices by other agents. 


In this short comment, we will not discuss the technical properties of 
Dependence Modal Logic. An excellent analysis of the proposed logic is 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 255-263. 
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made by Merlijn Sevenster (2008). Sevenster proves that the expressive 
power of dependence modallogic does not exceed standard modallogic, but 
that dependence modal logic can express certain properties more succinctly 
(witnessed by the fact that adding the dependence atoms to the language in- 
creases the complexity from PSPACE to NEXP). Sevenster also proposes an 
attractive alternative notation for the dependence atoms: dep(xl, ... ,Xn ;Y) 
instead of =(XI, ... , Xn, y). 

Rather than going into technicalities, we will try to discuss more gen- 
erally what the dependence atoms, which one can see as declamtions of 
dependenee between variables (‘attributes’) or propositions (‘facts’), do and 
do not express. 

Actually, it was the request to comment on this work at the KNAW- 
workshop New perspeciioes on Games and Interaction that made me realize 
that the notion of dependence as defined in these atoms does not always 
coincide with common uses of the notion of dependence. Before the KNAW 
workshop, I wanted to prepare my oral comment to Professor Väänänen's 
presentation. I was convineed that what I was going to say, should depend 
on what he would talk about, how could it otherwise count as a comment? 
So, I prepared my comment only aftel' carefully reading the material sent 
to me, and left openings for things that would come up during the actual 
talk. 

But in the break before the talk, a more experienced speaker confided me 
that "if you are asked to comment on somebody's work, just talk about your 
own work." At first, I was confused by this advice, because it confticted with 
the dependence I sensed in the concept of 'comment'. But then I realized 
that this was in Iact LoLally consistent wiLh Lhe mat.hernat.ical Lheory of 
dependence presented by Väänänen: I could just have prepared a fixed 
talk about my own work, even without reading the paper. According to 
Dependence Logic my comment would depend only on Jouko's talk: the 
comment would not depend on anything (would be constant), therefore it 
would depend on anything! 


Fact 1. For any propositions p, q, qi, ...,qn: if =(p) is satisfied in a set of 
nodes, then so is =(q,p), and more generally: =(ql, ... , gn,p). 


The conclusion could be that the dependence implicit in the notion of 
‘to comment’, is not just a functional dependence, but maybe a stronger 
sense of dependence. If you want to express that a different talk should 
amount to a different comment, one should add an injectivity condition on 
the function that establishes the dependence: With the usual dependence 
atoms (on variables) we have: =(XI, ... , Xn, y) is true for a set of valuations 
(a team) if y = f(Xl,...,Xn) for some f But it is not necessarily the 
case that (XI, ... , Xn) A (#},..., 2) implies FXL ...,xn) A f(xi,---, £h). 
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Adding injectivity condition would express a stronger property: that the 
talk deiermines the comment (which may be too strong actually for this 
application). 

This discussion shows that the "common phenomenon" that dependence 
is (quoting Väänänen), is not so easy to capture mathematically. In this 
comment we explore the kind of dependence expressed in the dependence 
atoms as proposed for Dependence (Modal) Logic a bit further. 


2 What is dependence? 


In order to clarify a bit what it could possibly mean, we simply retrieve 
the 'common' meaning(s) of the word dependenee from Merriam-Webster's 
dictionary: 


dependence 

1: the quality or state of being dependent; especially: the quality or 
state of being influenced or determined by or subject to another 2: 
reliance, trust 3: one that is relied on 4 a: drug addiction (developed 
a dependence on painkillers) b: habituation 


If one proposes a mathematical theory of dependence, it is good to spec- 
ify which sense of dependence one intends to formalize. It is deal’ from 
the contexts mentioned in the quote above from (Väänänen, 2008), that 
for example ontological dependence, as in "a child is dependent on his par- 
ents" (sense 2 and 3) falls outside the scope of Dependence Logic. (For an 
overview of theories of ontological dependence, see Lowe, 2005.) Obviously, 
also dependence in the sense of addiction (4) is not within the intended 
scope of Dependence Logic. The dependence atoms = (Q)I,...,qn, p) are 
declarations of dependenee in the sense of 1: one fact is being deter- 
mined by other facts, or for variables: the value of one variabie is somehow 
determined by, or at least correlated with the values of other variables. But 
still one can wonder what we mean exactly by that. 


3 A few interesting propositions 


In Dependence Modallogic, the dependence atoms will be evaluated in a set 
of nodes. The modal aspect of the logic is that this set of nodes evolves along 
the accessibility relations in the Kripke model, by taking the modalities 
as operating on sets of nodes, starting from the set containing the actual 
node (or nodes). For the evaluation of the dependence atom, it is only the 
corresponding set of valuations for the propositional variables that matters. 
In fact, the dependence in Dependence Modal Logic is a propositional rather 
than a modal dependence (this in contrast to the independenee in IF-modal 
logics as discussed in Tulenheimo and Sevenster, 2006). 


We discuss some (propositional) facts in Dependence Modal Logic. 
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3.1 Non-idempotency and bivalence 


Let's look at the non-idempotency phenomenon pointed out in section 3 of 
(Väänänen, 2008): the formula C := O(=(p)V =(p)) is shown to be true in 
the given model N (ibid. Figure 4), while B := Op is not. 

Note that this does not depend on the chosen model N, nor on the 
O-modality. In fact, for any propositional variabie p in any set of nodes 
(valuations) W: W = (=(p)V =(p)), while W = =(p) only holds if all 
valuations in W agree on p. It is easy to see that this fact ‘expresses’ the 
two-valuedness of the underlying propositionallogic. 

Compare this with (first order) Dependence Logic, where = (X) is true 
for sets of valuations that give X a constant value, (=(X)V =(x)) for sets 
that give X at most 2 values, (=(x)V =(x)V =(x)) at most 3, and so forth. 
To count the number of elements in a model, we need to put a universal 
quantifier in front (e.g, Vz|=(x)V =(X)] is true only in models containing 
at most two elements). This is a difference with the two-valuedness of the 
propositional part of modal logic: this does not depend on aspecific Kripke 
model, making the O-modality in a sense irrelevant. 


3.2 Does a consequence depend on its cause? 


What could be the formal relation between causation and dependence: if 
one fact causes the other, does the one then also depend on the other? 
In the absence of a causation-connective, despite the fact that causation 
and (material) implication are notoriously different, we do a first test by 
investigating the following implicational formula. Does for each set of nodes 
W 


ie apati. 


The answer can be seen to be 'no': take 


W := {(po;7p1), (po; 771); (po; p1)} 


(where we identify the three nodes with their valuations for Po and pı). If 
we write out the implications, the question boils down to 


9 


WE Gpo Api)V =(pO,PL). 


We split W at the disjunction. Only the subset /{(po;'P1)} satisfies the 
first disjunct, so {(,po; Pl), (,po; —p,)} should satisfy the second. But it 
doesn't, because p: gets a different truth value despite po's truth value 
being the same. 

However, note that both: 


E (po pi) > =(po, pı) 
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(po + pi) > =(P1, po} 
If two atomie propositions are equivalent, then the truth value of the one 


is a boolean function of the truth value of the other: it is the same. So, 
equivalent formulas are always mutually dependent! 


3.3 Axioms for functional dependenee 


One thing a mathematical theory of dependenee could be able to bring, is 
an axiomatization of the notion of dependence. Armstrong's axioms from 
database theory, are known to be sound and complete for functional depen- 
dence. They are actually not formulated in terms of a single attribute that 
may depend on a sequence of attributes, but in terms of sets of attributes. 
If we write =(X, Y) for 'two database records that agree in the attributes 
in X, also agree in the attributes of Y', they are: 


Al If Y CX then =(X, Y) (trivial dependenee or Reflexivity) 
A2 If =(X, Y) then =(X UZ, Y U Z) (Augmentation) 
A3 if =(X, Y) and =(Y, Z) then =(X, Z) (Transitivity) 


They are refiected in the following rules for the dependenee atoms in DL 
(cf. also Väänänen, 2008, Example JA, 10-12): 


1. =(x, x) (trivial dependenee or refiexivity) 

2. If =(x, z) then =(x, y, z) (augmentation) 

3. if=(x,y) and =(y,z) then =(x,z) (transitivity) 

4. If =(x, y, z) then =(y, x, z) (dependenee is order irrelevant) 
5. If =(x, x, y) then =(x,y) (dependence is resource insensitive) 


Of course, these are axiom schemes in the sense that they should be general- 
ized to arbitrary numbers of variables (e.g., also =(x, y, t, z) >=(2,t, y, Z)). 
Rules 4 and 5 now appeal’ because of the switch from sets to sequences. 

These axioms are really for functional dependence, and their soundness 
can be easily checked by writing them out in terms of the existence of 
functions. However, if we informally read the dependenee atoms as ‘depends 
(only) on’, not all of them sound completely natura!. For example, is it true 
that every attribute depends only on itself? And also, the augmentation rule 
2 amounts to the paradox of Fact 1: a constant (an attribute for which =(x)) 
depends on nothing else, but at the same time it depends on anything else (= 
(Yl, ... , Yn, X)). These considerations make us aware that Dependenee Logic 
in fact remains a mathematical theory of the already quite mathematical 
functional sense of dependence. 
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4 Dependenee versus independenee 


We make some remarks on the relationship between dependenee and in- 
dependence, which is the central concept in Independenee Friendly Logic 
(Hintikka, 1996). Instead of declaring dependenee in an atomie proposition, 
the independenee in IF-Iogic is declared at the quantifier: one lists the at- 
tributes on which some attribute is not supposed to depend, leaving the 
attributes on which it may depend to be determined by the context. This 
context is formed by both the other quantifiers within the formula, but also 
by the domain of the set of valuations (the 'team') for which the formula is 
evaluated. A detailed study of the effects of this latter part of the context 
can be found in (Caicedo et al., 2007), where it is shown how equivalence for 
open formulas can only be soundly defined by fixing a context of variables. 
We note that the translation procedure from IF-Iogic to Dependenee Logic 
for sentences, given in (Väänänen, 2007, p. 46), does not work for open 
formulas for this reason, as the following example shows. 


Example 2. Consider the IF-formula y Ay /2|x y]. The translation 
into DL would be y* := dyl[=(y)A x = y]. This formula is only true with 
respect to sets of valuations in which the value assigned to x is constant 
(then we can extend every valuation with the same constant assignment to 
y, thereby both satisfying =(y) and x = y). However, this is different for 
the original IF-formula y. Consider the set of valuations V consisting of 
vi = (Xt O,Z => 0) and v2 = (xt 1,Z »> 1). This is a set in which x is 
assigned two different values, hence it does not satisfy y*. It does however 
satisfy the IF-formula y, because we can let the value we assign to y depend 
on (by making it equal to) the value of z. 


To go from saying 'y must be independent of x’ to 'y may depend on 
the variables other than x’, one needs to be able to determine the set X of 
all variables that y could possibly depend on. For open formulas, this set 
consists not only of variables occurring in the formula, but also on variables 
in the domain of the valuations that may not occur in the formula itself 
(like z in the example). 

In conneetion to the example, it is interesting to note that rule 3 from 
the previous section formalizes the issue that makes the validity of Hodges' 
(1997) formula Yz3z Ay ja |e = y] from IF-Iogic counterintuitive. On the 
one hand, y is declared independent only from x, not excluding that it may 
depend on Z. But Z may depend on x, and then for y to depend on Z implies 
to depend on x by transitivity. This apparently violates the declaration of 
independenee from x. Dependenee Logic makes this nicely explicit: 


Vedzdyl=(x, 2) =(z, y) NX = y]. 


Indeed, it follows from the transitivity rule that y also depends on x in this 
formula. Note that adding an extra conjunction to this translation, trying 
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to enforce the independenee requirement explicitly, does make the formula 
false, by contradiction: 


Vadzdy[=(z, 2)A =(z,y) A (7 =(x, y)) Ax = yl. 


But not always do we get such clean-cut contradiction from a negated de- 
pendenee atom. As noted in section 5.2 of (Sevenster, 2008), it is not really 
the case that negations of dependenee atoms declare independence. In fact, 
a negated dependenee atom is only true in the empty set of nodes or valu- 
ations, hence will not in itself express independenee in non-empty sets. In 
order to express independence, one needs to jump to the meta-level of second 
order logic (to express that there is no function witnessing the dependence). 

Dependenee and independenee are duals, but not really opposites in the 
contradictory sense. Therefore, it is not straightforward to make the switch 
from declaring dependenee to declaring independence. 


5 Conclusion 


There are several reasons why the topic of Dependenee Logic is at home 
within the theme New perspectioes on Games and Interaction. There is a 
new perspeciuoe in the focus on dependenee rather than independence, as in 
IF-logic. 

Like IF-logic, the mathematical theory of dependenee comes with a game 
theoretical semantics. Where IF-logic enlarged the field of logical games 
with games of imperfect information, Dependenee Logic adds a uniformity 
condition on the winning strategies. This is a less standard generalization, 
in the sense that it is not part of standerd game terminology. The cor- 
respondence with database theory, and correlations between attributes is 
more convincing in our taste. With respect to the interaction part: de- 
pendenee can be seen as a form of interaction between facts. Note that the 
concept of (in)dependence does not arise in isolation (cf. Hodges, 2007). 

The main conclusion aftel' considering several Dependenee modal formu- 
las, is that the notion of dependenee expressed in the dependenee atoms, 
is strictly functional dependence. It allows to talk about functional depen- 
denee while keeping the function implicit (compare, for example = (y, x) 
with x = g(y) and =(q,p) with p = (~q)). 

One can wonder what kind of dependenee functional dependenee actu- 
ally captures. It is a bit counterintuitive for a notion of dependenee to have 
things that are both dependent and independent on anything (viz. con- 
stants). It seems that functional dependenee expresses some kind of 'corre- 
lation', which we sense to be a weaker notion than dependence. 

But in the end, we think the historical (technical) evolution of logic 
provides the clearest view on the motivations for studying dependenee and 
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independenee between variables and propositions. Dependenee of one vari- 
able upon another is already a natural feature in logical languages, by the 
structure (nesting of quantifiers). The original spark to study dependenee 
and independenee was the generalizations of the specific syntactic pattern 
built in in first order logic (‘Frege's fallacy’ Hintikka, 1996), and to see how 
they would behave and extend the expressive power of the language. A 
great motivational insight was Hintikka's idea to link the dependenee of 
variables to availability of information in semantic games, and thereby in- 
dependenee to imperfect information. But as many examples have shown 
(Janssen, 2002; Hodges, 1997), interpretation of the IF-formulas naturally 
in terrns of what one generally understands as ‘independence’, is not so 
straightforward. 

Syntactic subtieties in the end turn out to be important spoilers for 
smooth and elegant results. This is shown in (Caicedo et al., 2007) by the 
amount of work we need there in order to restore the Prenex Form Theorem 
for IF-Iogic. But using this syntactic result, we are able to make a syntactic 
gain for first order logic: by translating a first order sentence into it's IF- 
prenex form, and then skolemizing it, we avoid unneccesary arguments in 
the Skolem functions. With these final remarks, I have managed to follow 
the advice on commenting: to speak about my own work. 
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Abstract 


We provide a syntactie framework for analyzing extensive-form games. 
Specifically, we correlate to every such game a "language" and axiom- 
atization. The language and the axiomatization, are presented as a 
class of sound and complete models, through which we explore the 
epistemic conditions for the reduction process introduced by Pearce 
(1984) and Battigalli (1997). 


1 Introduction 


Economists and game theorists use concepts of knowiedge, belief, and ra- 
tionality to characterize solution concepts. Inthis work we present an epis- 
temic characterization of the red uction process introduced by Pearce (1984). 
To do this, we use a syntactic environment in a way that correlates a lan- 
guage to every extensive-form game. 

A natural starting point is Aumann's work on backward induction (Au- 
mann, 1995). This work uses the usual knowledge partition setup for ana- 
Iyzing interactive rationality in generic perfect information (PI) games. 


1.1 Aumann's model 


Let f be a PI game and let Q be a set of states of the world such that every 
player i E / is equipped with an equivalence relation that forms a partition 
of Q. Knowledge is derived from this partition in the usual way. Aumann 
correlates a strategy to every player i in each state WE Q si(W) such that 
every player knows his own strategy in w. 

Civen' h« Hi, player i is defined to be h-rational in state w E Q if it is 
noi the case that he knows he has a better strategy in the sub-game that 
starts in h than Si(W). Player i is rational if he is h-rational at each and 


1 H; is the set of non-terminal histories in which player i is required to perform an 
action. 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 265-281. 
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FIGURE 1. 


every history h E Hi. Rationality is obtained if all players are rational. The 
main result of Aumann's work states the following: 


Theorem 1.1. In generic PI games, common knowledge of rationality is 
consistent with and entails the backward induction strategy for every player. 


That is, if in a state of the world w E Q all players are rational, all 
players know that they are all rational, and so on ad infinitum, then all 
players play according to their backward induction strategy. 

Aumann's seminal paper provided a formal logical characterization for 
the solution concept of backward induction and also gave rise to a fruitful 
debate on the nature of this solution concept in PI games. 

The main critiques ofthe paper, as weil as ofbackward induction analysis 
in general, can be explained using the following example: In the game 
depicted in Figure 1, Ann's dominating action is to exit on her first move. 
If for some reason Ann stays, then she should be interpreted by Bob as 
irrational. The event "Bob is called to play" contradiets Ann's rationality 
and, in particular, there is no reason to assume that he will stick to his 
backward induction strategy. That is, the demand for common knowledge 
of rationality in eoeru subgame seerns awkward since it does not take into 
consideration the moves that lead to the history. 

The problem, as we see it, sterns from the fact that the assumption of 
interactive rationality in the model ignores past history. A player does not 
update his knowledge (or beliefs) in the course of the game with respect 
to a possible deviation. The reason for that is that the model does not 
take into account the interactive nature of the game and does not allow for 
counterfactual reasoning. 

To circumvent this inconsistency we are going to weaken some restrictive 
assumptions in Aumann's model. First, the language to be constructed 
includes a belief with probability one operator (rather than knowiedge) for 
every player / E J in every one of his histories h E Hs, and the belief revision 
is carried out using Bayes rule. 

Second, we replace the interactive knowledge operator by a sironq belief 
operator. Player / strongly believes in formula 9 if he believes in 9 at each 
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and every one of his histories that are logically consistent with g. Using this 
forward-induetion reasoning eliminates the consistency problem. 

This work has some similarities with (Battigalli and Siniscalchi, 2002). 
Battigalli and Siniscalchi assign to every game in extensive form a mathe- 
matical object called a belief-complete type space. This object includes all 
possible Bayesian updating belief hierarchies that satisfy some coherence 
conditions. As they show in their paper, the use of belief-complete type 
space is crucial in conducting a comprehensive analysis independently of a 
specific belief type space. 

By using a belief with probability one operator rather than probabil- 
ity measures we avoid the complication involved in constructing a belief- 
complete type space (see Battigalli and Siniscalchi, 1999). Moreover, the 
use of syntax and logical consistency replace the completeness demand in 
the type space construction and makes the analysis much more transparent. 


2 Framework 


Start with a generic PI game/ G. The game consists of: player i's histories 
Hi, i.e, histories in which player i is active and the empty history, for each 
player i; terminal histories Z; and a payoff to each player at each terminal 
history. A stTategy for i is a function that assigns to each of i's histories 
h E H; an action at h. Each strategy si of i determines a set H( Si) of 
histories of i, namely, those that si allows (does not preclude by an action 
at a previous history). A plan of i is the restriction’ to H (Si) of a strategy Si. 

We now construct a formal language whose building blocks are the fol- 
lowing: 


Atomie sentences. 
These have the form "playeri uses plan Pi," denoted simply Pi. 


Left parentheses and right parentheses. 


Connectives and operators of the propositional calculus. 
As is known, it is sufheient to take just "or" (V) and "not" (~) as 
primitives and in terms of them to define "and" (1\) and "implies" (—). 


Belief operator. 
For each player i and history h E Hs, there is a conditional proba- 
bility one belief operator, denoted bh. Informally, bh9 means that if 
player i were to observe history h, he would ascribe probability one to 
g. Players are not permitted to condition on their own actions. We 
will return to this demand when we represent the axiomatization. 


2 For simplicity we introduce the framewerk for PI games, but in fact our results hold 
for a more general case to be discussed in Section 7. 

3 Plans are sometimes called "strategies". Here we do not want a strategy to be defined 
at the histories that it excludes. 
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A formula is a finite string obtained by applying the following two rules in 
some order finitely often: 


e Every atomie sentenee is a formula. 


e If f and gare formulas, so are (J) V (g), ,(J) and bh(J) for every 
non-terminal history A. 


The set of all formulas, ealled syntax, is denoted by X (for the game under 
eonsideration). 

Let hand hl be histories, where h > hl means that h follows hl in 
the game tree (or that hl is a prefix of h). Let a be an action at history 
h « H; and "i plays a” (or a for short) is a formula, namely VPi, where the 
disjunction ranges over all plans of i that either preclude h or eall for him 
to play a at h. Also, "h is reaehed" (or simply h) is a formula, namely Ad, 
where the conjunction ranges over all players j with histories on the path 
to h, and at those histories, over all those actions d leading to h. If L is a 
set of histories, then "L is reaehed" (or simply L) is the formula Vh, where 
the disjunction ranges over all REL. 

Let h « H, be an h-plan of i that allows h, and denote by Pi(h) the set 
of i's h plans. An opposition h-plan is a conjunction of plans that allow h, 
one for eaeh player other than i. An h-plan Pi together with an opposition 
h-plan P-i determine a terminal history of the game tree z, where z > h 
and a payoff u;(Pi,p-i) for i. The set of all opposition h-plans is denoted 
by P-i(h) and the formula eorresponds to "all players other than i play 


aeeording to h" is: 
h0 = V P-i- 


p-iEP-i(h) 
2.1 Inference 


To make our language meaningful we provide an axiomatization that will 
deseribe the game at hand. We start by introdueing the axioms and rules. 
Here f, 9 represent formulas and h, h histories of the same player: 


T (1) 

Vpr; where the disjunction is over all plans of player i. (2.1) 

'(Pi A qi) where Pi and qi are different plans of the same player i. (2.2) 
bh(J = g) > bh f = bh9 (Kj. (3.1) 

bhf —,bh ,f (Dj. (3.2) 

b’ f > b? b’ i, where h, h « Hi. (3.3) 

„bhf sph apt (3.4) 


Pi > bh Pi for every h « tt; (3.5) 
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bh ha. (3.6) 

(bh i NAb? sh?) => bê il, where h < i: (3.7) 
From Í — 9 and 1 infer 9 (modus ponens). (4.1) 
From 1 infer bh i (generalization). (4.2) 


Axiom (1) includes all the tautologies, that is, formulas that assess the value 
true in every possible truth valuation. Axioms (2.1) and (2.2) correspond 
to the obvious demand that every player execute exactly one plan. Axioms 
(3.1) and (3.2) represent classical modal beliefaxioms. Axioms schema (3.3) 
through (3.5) combine versions of the "truth" and "introspection" axioms. 
Briefly, they say that players are sure of their own plans and beliefs. 

bh is interpreted as the belief of i at history h regarding the plans and 
beliefs of players other than himself. Therefore it makes sense to require that 
player i in h believes that players other than himself played in accordance 
with h, that is, bh hO. 

Axiom (3.7), which deals with the belief revision policy, states that if 
i believed i at h, and also believed that h° "could" occur at h, then he 
believes 7 at Í It refiects the fact that players update their beliefs in a 
Bayesian way. 

(4.1) and (4.2) are the two inference rules of the axiom system AX. A 
proof in AX is a finite sequence of formulas each of which is either an axiom 
or follows from those preceding it through the application of one of the two 
inference rules. A proof of a formula I is a proof whose last formula is 1. 
i is provable in AX if there exists a proof of Í written l-Ax f. 

i is inconsistent if its negation is provable; otherwise it is consistent. 
Formulas f4, fo, ... are inconsistent if the conjunction of some finite subset 
of them is inconsistent; otherwise they are consistent. They entail 9 if the 
conjunction of some finite subset of them entails g. 


3 The theorem 
3.1 Rationality 


Instead of defining the rationality of a player in a particular history as 
in (Aumann, 1995), we define rationality in terms of the plan that a player 
uses and his beliefs. We replace the utility maximization demand by a 
weaker one; namely, we require that in every optional active history dictated 
by the player's plan, it not be the case that he believes that he has a better 
plan. 

Formally, we say that i uses rationol plan Pi if in every history h E 
Hj (pi )4 it is not the case that he believes that he has a better h-plan. A 


4 We denote by Hi(Pi) the set of player i's histories that are not precluded by the plan Pi. 
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formula that represents the rationality of a player of course needs to take 
into account player i's payoff function in the game. 

For each pair of different plans of player i narnely, Pi and qi, we denote 
by oF. (qi) the disjunction of opposition h-plans in P_i(h), where qi yields a 
higher payoff’ than Pi. The formula that represents that plan Pi is rational 
for i is the fol!owing: 


T(Pi) = Pi > A N x bh Qh. (4). 


{hlhEH(Pi)} {GE Pi(h)|\ae#pa} 


Define player i to be rational if he uses a rational plan. That is, 


Ti Nri). 


Pi 


In brief, a player is rational if it is not the case that he believes that he has 
a better plan in one of the histories that is consistent with his plan. This 
is in fact a much weaker demand than in (Aumann, 1995), where a player 
is required not to know that he has a better strategy in eoeru one of his 
histories. 


Remark. We do not claim that the above formal definition of "rational- 
ity" is the only possible one. We do however claim that any reasonable 
definition entails the fol!owing: ifi is "rational" in any commonly accepted 
sense (such as utility maximization), then certainly Ti obtains. The formula 
corresponding to al! players being rational is 


r= Nr 


3.2 Strong belief 


The strong belief operator is a substitutional concept for interactive common 
knowiedge. The operator is external to the language and it is defined in such 
a way as to resolve the consistency problem discussed in the Introduction. 

We say that i sironqls; believes a formula gif, for each of i's histories 
h « U., either 


(i) i believes 9 at h, or 


(ii) 9 precludes h being reached, or, equivalently, 9 is inconsistent with A. 


5 Le., Qh (au) = V{P-i E P_i(h) |Ui(qi,P-i) > Ui(Pi,P-i)}; if there are na such P-i 
we set Qh (qi) =L. 
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In words, i continues to believe 9 no matter what happens, unless he reaches 
a history that is logically impossible under g. We can write it as a formula 
in the following way:" 


Sbi(g) = A bh(s). 


{hEH; |\'/ax-7(hAg)} 


We say that 9 is stmngly believed (or that there is sirong belief of g, 
written sb(g)) if each player strongly believes g. Mutual stronq belief of 9 
of order n (written sbn(g)) is defined as sb(sb’~ 19?’ that is, each iteration 
provides for strong belief of the foregoing iteration (note that the strong 
belief operator does not commute with conjunction). Common sirong belief 
of 9 comprises all the formulas sb" (g) for all n. 


The main result of this paper states the following: 


Main Theorem. Common strong belief of rationality is consistent and 
entails the backward induction outcome in every PI game. 


4 Soundness and completeness 


Before proving the main theorem, we would like to present a class of simple 
models for our axiomatization that links the syntax to the semantics. The 
most preferabie way would be to link the syntax to a class of models that 
characterize it by soundness and completeness relation. The way to do that 
would be by looking at the canonical model of the language with respect to 
the logic that our axiomatization defines. 
We would first like to introduce some more terminology: 

An axiom system Ax is said to be sound for a language Š with respect to 
a class of models C if every provable formula f is valid with respect to C. 
An axiom system Ax is said to be complete for a language Š with respect 
to a class of models C if every valid formula f with respect to C is provable 
in Ax. 


4.1 The canonical model 


Definition 4.1. A set of formulas f is maximal consistent with respect to 
Ax if it satisfies the following two conditions: 


a. F is consistent with respect to Ax. 
b. f is maximal with respect to that property. 


It can be seen that maximal sets do exist’ and satisfy the following 
properties: 


6 If there are no h E H; that are consistent with g, put sbi(g) = L. 
7 See (Chellas, 1984). 
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1. ris closed under modus ponens (4.1). 

2. f contains all the theorems ofAx. 

3. For every formula y, pEr or ~g Er. 

4. For every formula y, Y, pV YEr i/"pEr or WEr. 
5. For every formula y, Y, pA% Er if" pE rand Er. 


6. Every consistent set of formulas can be extended to a maximal con- 
sistent set. 


Now let Q be the set of all maximal consistent sets; we call the elements of 
Q states of the uiorld. For each r E Q and non-terminal history h « Hi, we 
define 71h to be the set of all formulas that player i h-believes in r. That 
is, 
T/h = {p |b” y Er}. 

For every player i and a non-terminal history h E Hi, define the usual 
accessibility binary relation Rh over Q as follows: let r, A E Q, r RhA iff 
rlh CA. Let Be be the set of all states of the world that player i considers 
possible at h E Hi, that is, 


BR = {A E Q | TRA}. 
Observation 4.2. rl h satisfies the following conditions: 
1. rh is consistent (therefore Bes 0). 
2. rlh is closed under (4.1) and (4.2) (if y Er, then bhy ET). 
3. pE rl h for every y such that I-AXx y. 


Proo]. Part 2 follows from positive introspection, while part 3 is straight- 
forward from generalization. For part 1, assume by way of contradiction 
that “lh is not consistent. Then we have Yı; Yr E rlh such that 
AX l- A(yi A... Nk). By definition, bh pı, bhy, E rand so from 
K we get b"(wi A... Apk) Er but from part 3 bh(y, A... A pk) Er, a 
contradiction to D. Q.E.D. 


Let r E Q; we would now like inductively to define a truth assessment 
over the set 2, where ||y|| is the set of states in which y holds: 


e for atomie sentences, r = Pi ij}" Pi E l; 


e for formula y, r H ~g ij" r Fy; 


e for formulas y and Y, r FeV i"r Evorr FY; 
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e for formula y = b” 4, k € Hi, PE b” gy iff BE C lly]. 


Proposition 4.3. For every Ý EQ and every formula vy, - y ijl p Er. 


Proo]. We wili prove the proposition using induction on the depth of the 
formula. 

For formulas of depth zero the proof is immediate from the properties 
of maximal consistent sets and the truth assessment policy. We prove the 
proposition first for formulas of the form y = b” w, where p is from depth 
n-1 >20. The general case foliows from the properties of maximal consistent 
sets and the truth assessment policy. 


<: Ify Er then by definition of r/h, w Er/h; therefore p EA for every 
A EB} by the induction hypothesis BE < ||%]|; therefore r ]- ¢. 


=>: If f H ọ then BÈ C |l] so 4% E A for every A such that r/h C f; 
therefore f/h|-Ax % for otherwise we could have constructed a maximal 
consistent set A’ such that f/ hu {~y} < A’. But because F / A contains ali 
the theorems of AX and is closed under 4.1 and 4.2 we get that 7 E r/h 
and therefore y E f. Q.E.D. 


Thus Proposition 4.3 leads to the foliowing theorem: 
Theorem 4.4. Q is a sound and complete model with respect to AX. 
We would like to state a few properties of the set BŁ. 


Lemma 4.5. For every f E Q player i, and history h E Hi, B} satisfies 
Lhe foliowing properties: 


1. BÈ Zo. 
2. Let h Ett, such that h > h and BEA IIŚ # 0; then Bh = Bh ah]. 


Proo]. Part 1 is a consequence of part 1 in the previous observation. For 
part 2, assume BEN lack Æ 0; then from the truth assessment policy we 
get f H -ib" „ha, but by Lemma (4.1) that means that ab” „ha Er. ıf 
for some formula j, bhj E f, then, because f is a maximal consistent set, 
bhj \ ,bh—he Er; therefore from (3.6) and (4.2) b' tc t: 

We show the foliowing: if bhj Er then bê j Er. Therefore /h<r/Î 
and in particular BÈ C BÈ. This foliows from the fact that if r/h C A then 
obviously P/h C A. From Bh C ||h°|] (3.4) we get the desired result. Q.E.D. 


We would like to consider a class of models with the following properties 
for our game G. Let Q be a set of states of the world. Each player i is 
equipped with an equivalence relation ~; over Q and a plan Pi = Pi(W), for 
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each wE Q. For sa equivalence class Mi(w) and history h E Hi, consider 
a nonempty set B} M: (w) C Q. Let llpi | be the event in which player i executes 
the plan Pi. We would like the following properties to hold in the model: 


1. If W~; io’ then Pi(W) = Pi(W'). 
2. Bhi) < Hi(w) n [nots 


3. For h,h E tt, such that h > h, if B}, (w) N lholl 4 0 then Biia C 
Bh (uw) n llholi. 


We can think of the equivalence relation ~; over Q as arelation that defines 
knowiedge. That is, w ~; wı if player i cannot distinguish between these 
two states. The first requirement represents the demand that each player 
knows his own strategy. 

The second requirement entails that each player know his beliefs and 
that if the game gets to history h E H; player i wil! assign probability one 
to the set in which players other than himself played in accordance with A. 
The third requirement relates to the belief revision policy. 

We cal! the above class of models for the game C, M(C). The truth 
assessment in those models is as in the canonical case. We are now in a 
position to state the main result of this chapter: 


Theorem 4.6. The class of modeIs M(C) is sound and complete with 
respect to Ax. 


Prooj]. The soundness part is omitted. Completeness is a straightforward 
consequence of Lemma 3.1 and Proposition 1. It suffices to note that the 
canonical model satisfies al! the above requirements. The only thing left 
to define is the equivalence relation for every player i. Let r, |" E Q. We 
define r ~; I" if, for some history h E H; r/h = r’ h, we have to show 
that B} depends only in the equivalence class of r. But it turns out to be 
pretty clear, assume that r/h = r l/h for some h E H; and let hl E Hi. 


If f E r/hi then bh f E rand from the positive introspeetion property 


(3.3) b’ b” f Er it follows that b’ b” f E I". Therefore b” f E I" and 


f E rl/hl, and vice versa. Therefore r/ hl = ri/hl for every history hl E Hs, 
and so Bk = BE for every history hl E Hi. Q.E.D. 


The set Bito) is the set of states that player i considers possible in h, 
or, equivalently, the support of player i's measure in h. From property 3 
we can see that the belief update is not strictly Bayesian; that is, we have 


8 Ill is the event in Q where players other than i play in accordance with history h. 
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BE as < Bh (ws) n Ilholl rather than equality. If we want strict Bayesian 
updating we must add the following axiom: 


bê f = b*(f v =°) where h < À. (3.8) 


Denote by Ax! the axiom system Ax with the addition of (3.8) and by 
M+(G) the class of models with equality in property 3. We then get the 
foliowing theorem: 


Theorem 4.7. M+(G) is sound and complete with respect to Ax". 


Proo]. It suffices to note that in the canonical model axiom (3.8) entails 
that Bh 2 Be n llholl for every f E Q. Assume by way of contradiction 
that Bh Ž Ben | het then we have A E Q such that AE Bn lhol but 
Ag BÈ. By definition of Bh, T/h Z A and so there exists f such that 
bê f E rand f ¢ A and so from maximality of A , FE A. Then, from (3.8), 


b (J V HO) E rand so f V,ho E A but hO E A, a contradiction to the 
consistency of A. Q.E.D. 


5 Proof of the Main Theorem 


Consider the foliowing inductively defined series of plans: For every player 
i, let pe = Pi, Pa, = Thy PP and pa = Tare Let n > 0 and assume 
pr] to be defined for every player i, and let Hn-l be those non-terminal 
histories that are reachable by profiles of plans from P'r:', Now Pi E P” if 


it satisfies the following requirement: 
: Sq 
1. Pi ER's" 


2. For every history h « H(pi) N Hn-/ there exists P-i E P”; '(h) such 
that 2: (p;, p_i) > uilqai,p_i) for every gi E Pi(h). 
In words, a plan Pi E pied is in P” if and only if, for every non-terminal 
history h that is consistent with P” and Pi, there exists some opposition 
h-plan P-i E pre! such that Pi is a best reply to ali plans gi E Pi(h). 


Lemma 5.1. For every generic PI game there exists m such that, for every 
player i, P? = P;” for every n >m. And, every profile PE P’” leads to the 


t 
unique backward induction outcome. 


Proo]. See (Battigalii, 1996). Q.E.D. 


Lemma 5.2. A plan Pi of player i is consistent with the set of formulas 


{T, Sb(T), ... ,Sbn(T)} iff Pi E P+. 
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Proo]. We prove the lemma using induetion on n. 
Case n = O. 


=>: Let PiE P? sueh that Pi ¢ P}. By the definition of P} we have a history 
h E H; sueh that, for every opposition h-plan P-i, there exists qi E Pi(h) 
with wilpi, p—i) < wilq,p—i). Let a = pi(h), the action prescribed by Pi in 
h. We argue that there exists an action bÆ a in h sueh that the maximal 
payoff in T(h,a)9 is smaller than the minimal payoff in T(h,b)' Otherwise 
we eould find an appropriate opposition h-plan P-i sueh that Pi would be 
a best reply to P-i. Let qi E Pi(h) sueh that qi(h) = b. Then, for every 
opposition h-plan P-i E P-j we have w;(p;,p_i) < wi(qi,p_i). Therefore by 
the definition of Qh (ai) (the set of all the opposition h-plans sueh that, for 
player i, qi yields a higher payoff than Pi), we have Qh (qi) = h0. From the 
definition of rationality r we have l-Ax r A Pi > T(Pi) A Pi. But from the 
definition of T(Pi), FAx (T(Pi) A Pi) —> ~b” (QÈ (q:)), whieh together with 
axiom (3.6) contradiets (3.2). 


<: Let PE t», we have to show that P is consistent with 7. For every i, 
Pi E P}, therefore, by definition, for every history h E H(pi) we ean find 
an opposition h-plan, p; E P_i(h) sueh that u:(p:, p™;) > wilqi,p”,) for 
every h-plan qi. Let Q =- Trier Pi, where p ~; pi iff p; = p4. Therefore the 
equivalenee class Mi(w) is determined by player i's plan. 

Start with h « H(pi) and then put Be = (p:,p",). Now for the other 
h if h E H(pi) n H(p",), put Be - (Pi,p",), or else ehoose p”; and put 
Bh = (Pi,p",) and so forth until all the histories H(pi) are covered. For 
h g H(Pi) we ean let BR be arbitrary. We repeat the procedure for all the 
players. One ean see that our model belongs to the class M (G). 

To show that P — T, it would be suffieient to show that P | T(Pi) for 
every player i. la This follows from the faet that for every history h E H(pi) 
and qi E Pi(h), p*; € Qh,(q). 

Case n > O. 

At this point we have the basis for the induetion. Now assume that the 

theorem is obtained for every 0 < k <n. We prove it for n as follows. 


=>: Let Pi be a plan of player i sueh that Pi ¢ PP +’. If there exists a 
k< n- 1 sueh that Pi ¢ Pik+', then by the induetion hypothesis we are 
done. Assume that Pi E P”. Then for some history h E H(pi) n H" and 
for every P-i E P”, n P-i(h) there exists a strategy qi E Pi(h) sueh that 
ui(pi,p—i) < uilqi, P<). Denote (a = pi(h)); as in the n = 0 case there 
exists an action bin h sueh that the maximal payoff for player i in Tha) is 


9 The subtree Th corresponds to the subgame starting at history bk. 
10 Obviously for q; Æ Pi, P | r(qi)' 
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smaller than the minimal payoff in (h By Set r” = Ao<g<n sb" r (where 
sb°r = r); by the induction hypothesis the only plans of players other than 
i that are consistent with r”- ‘ are those in P”, and therefore l- Ax r” - g 
Veo Benes? h « H"; so by the definition of sb"> = sb(sbn-1r) we have: 


Hay Sbnr — bh(sbn-ly} 


Therefore l- ax r” — bh[V,,_,<pn, p-i] denote by P? (h) the set of plans of 
player i in Př that are consistent with the history h. By axioms (3.4) and 
(3.5) we have Fax r” — bh[V P”,(h)]. Let gi be a strategy of player i that 
allows hand qi(h) = b. So from both the assumption and the definition 
we get P™,(h) C Qh (a) and Fax PiAr(pi) > —bh[V Qh. (a)l Therefore 
l-ax Pi A r(pi) > abh[V P™,(h)]. Thus, Piis not consistent with r. 

<: Let PE pn+l. We have to show that Pis consistent with [r; ... ,sbn(r)}. 
From the fact that Pi E pitL as in the n = O case, for every history 


h E H; N H” there exists an opposition h-plan pi such that the following 
properties obtain: 


1. pe, E PE. 
2. ulpi,p”,) > ulg p”;) for every gi E Pi(h). 


Le Q- Di Pi; inductively we can construct Bt such that the following 
property will be satisfied: 


fork<n peP! spr". 


Now we can change B}, so that pi = (Pi, př) and obviously (*) would still 
be satisfied. By the induction hypothesis, forevery playeri and hE HnnH,, 


’ From the modification 


for the appropriate p, we get (Pi, pe.) Ee 7s 


of Be we have P | bh(r"- 1). But again using the induction hypothesis 


sb(r2- 1): Anen, bh(r"- ') and therefore P | sb"rand therefore P H 
re, Q.E.D. 


6 Battigalli and Siniscalchi 


In this section we present the model of Battigalli and Siniscalchi, and dis- 
cuss the conneetion between our model and theirs. We start with a few 
definitions related to their work. 

For a given measure space (X, X) and a nonempty collection of events 
B such that 0 ¢ B: 


11 The subtree J" is the subtree of the original game that is compatible with P": 
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Definition 6.1. A conditional probability system (CPS) on (X, X, B) is a 
mapping p(- |.) :Æ x B — [0,1] such that, for all B, CE Band A E ¥, (1) 
u(B IB) = 1, (2) u(- I B) is a probability measure on (X, X, B), and (3) 
A < B < C implies that p(A | B)p(B1C) = (A IC). 


The set of the conditional probability system on (X, X,B) can be re- 
garded as a (closed) subset of [A(¥)]5, denoted by A8 (Æ). 


Definition 6.2 (Ben-Porath). Given a (PI) game C, a type space on 
(H, S(-), T) is a tuple ¥ = (H, S(-), I, (Qi, Ti, gijicr) such that for every 


i E J, Ti is a compact topological space and 


1. Q; is a closed subset of Si x Ti such that projSi Q; = Si. 
2. gi= (gLh)hE'H :Ti > A™(Q_,) is a continuous mapping. 


For any i E J, gi,h(ti) denotes the beliefs of epistemic type ti conditional 
on h. 


Definition 6.3. A belief-complete type space on (H,S(-),!) is a type space 
£= (H,S(-), I, (Qi, Ti, gijicr) such that for every i E Z Q; = Si x Ti and 
the function gi maps Ti onto AH (jS; x Tj). 


Battigalli and Siniscalchi (1999) showed that for finite games, a belief- 
complete type space can always be constructed. 


6.1 Semantical conditional belief operators 

Let A; denote the sigma-algebra of events E C Q such that E = Q; x 
projo, E. A-i is similarly defined. The conditional belief operator for 
player i given history h E H is a map Bi,h : A-i — A; defined as follows: 


VEE A-i Bi n(E) {(s, t) EQ | gi n(ti)\(projo A E) 1}. 


Like the syntactical operator bf i}, the semantical operator Bi,h(E) has the 
meaning of "player iascribes probability one that his opponents' strategies 
and epistemic types are consistent with E, when he observes history h.” 

We say that player i strongly believes that an event E Æ 0 is true if and 
only if he is certain of E at all histories consistent with E. Formally, for any 
type space %, define the operator SB; : A-i — A; as follows: SB; (0) = 0 
and 

SBi(E) = n Bi,h(E) 
REH: EN[h]#2 


for all events E EA-i\ {0} and fhl is the event "history h occurs." 
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Along the lines of their syntactic approach, B&S define what they call 
"the mutual strong belief" operator. For any Borel subset E C Q such that 
E = IIF projo, E, let 


SB(E) = Nse: x projo, E). 
iEl 

To adjust this model to inclusion in the class M + (G), we have to specify a 
plan, an equivalence relation, and a set Bito) for every player i and history 
h « H; in each state of the world w. 

Obviously, for W= (S,t), Pi(W) would be the plan induced by the strategy 
Si. Define W~; WI iff ti(w) = ti(WÌ) and Si(W) = Si(W!- For every h E H; 
let Bhi) = (Si(W), ti(w)) x supp{gin(ti(w))}, where Si(W) and ti(w) are 
the strategy and the type of player i respectively in the state of the world 
wand supp{gin(ti(w))} is the support of the measure gi,h(ti(W)) in Q_;. 

It remains to show that the sets Bh ws) satisfy the requirement stated 
for the M + (G) modeis. 


Lemma 6.4. Let in E ff, such that a > h; if BÊ „n Ihol A 0 then 


Bh (ws) = Bhi cw) n Tholl . 


Proof. Ihol = Six S_i(h) x Tand, in particular, projg_,||h°|| = S_i(h) x 
T_i, is an open set. From the fact that Ph n Ihol = (ti(w), Si(W)) x 


supp{g; p(t:(w))} n Ihol #0 we get gih(ti(w))(UhON N 9—:) > 0 and by 
property 3 in definition 6.2 we get the result. Q.E.D. 


Theorem 6.5. The model obtained from the complete belief type space for 
a game is a canonical model with respect to.Ax" . 


Proof. The soundness part follows from the lemma; as for the complete- 
ness part, the proof is a bit more complicated and it relies heavily on the 
completeness of the type space. Q.E.D. 


7 Discussion 


One more important aspect in extensive-form games yet to be discussed 
counterfactual reasoning. To test the stability of an equilibrium concept, 
one must consider questions like "what would be the case if..." That is, to 
provide an epistemic foundation for a solution concept one must determine 
whether a player thinks that he could gain by deviating from the strategy 
prescribed by the equilibrium. 

Consider the following version of the centipede game (Figure 2). Back- 
ward induction reasoning leads to the profile of strategies in which Ann and 
Bob exit at each and every one of their decision points. But what if Ann 
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FIGURE 2. 


stays on her first move? Bob might interpret this move as irrational, in 
which case he would expeet Ann to be irrational also at her second decision 
point, and so he expects to gain by staying. Arm, by the above reasoning, 
thinks that by staying she might be interpreted as irrational by Bob who, as 
a rational player, will stay and so when reaching her second decision point 
she will exit and get 3 instead of I. 

This paradox can be resolved using Battigalli's best rationalization prin- 
ciple (Battigalli, 1996) which states that "players, beliefs conditional upon 
observing history h are consistent with the highest degree of strategie so- 
phistication of their opponents." That is, when a player observes deviation 
from an equilibrium path by another player, he assigns to that deviation 
the maximal degree of belief with rationality. 

The main theorem in fact states that the plans that are consistent with 
common strong belief of rationality are consistent with the principle that 
we have just described. That is, if Pi is a plan of i in a generic PI game that 
is consistent with common strong belief of rationality, then in every history 
h « H(Pi), Pi maximizes his payoff function according to the highest ratio- 
nality degree assumption. Formally, let h E H(Pi)' Assume that h is consis- 
tent with {T ... ,Sbm(T)} but not with {T, ... ,Sbm(T), Sb®+'C Then the 
reduction of the strategy Pi to the subgame starting at h maximizes player 
i's payoff function with respect to the assumption {b"(r), ... ,bh(sbm(T))}. 

Thus, if players stick to the best rationalization principle, and that is 
common strong belief, then no player will believe that he can gain by devi- 
ating from his plan. 


7.1 General extensive games 


We reduced the analysis to PI extensive-form games but in fact it is equally 
valid for general finite extensive games with perfect recall. The way to 
adjust the framework for this case is fairly obvious. Again we use plans 
rather than strategies, except that now H, is the collection of information 
sets of i; indeed, in the PI case it is identical to the set of histories of player i. 

The axiomatization stays the same but here we have a belief with a 
probability one operator for every player's information set rather than for 
every history. The rationality definition and the strong belief operator re- 
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main unchanged. The sound and complete models are the same once we 
replace histories with information sets. 

The way in which interactive rationality prunes strategies remains un- 
changed. That is, let p? = Pi and Př is inductively defined as before. Let 
Hn-l be those information sets that are reachable by profiles of plans from 
P'r:: PLE Pr iff Pi E Př! and for every information set h E A(pi)nHn-l 
there exists P-i E Pe such that ui(pi, Pi) > wi(qi, P—a) for every qi E 
Pi(h). Now we get the following version of (5.2): 


Lemma 7.1. A plan Pi of player i is consistent with the set of formulas 
{r, sb(r), ... ,sbn(r)} iff Pi E Pf’ +". 


In this case, unlike the generic PI case, there could be many optional 
outcomes in P?', 
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Abstract 

Boolean games are a logical setting for representing strategie games 
in a succinct way, taking advantage of the expressive power and con- 
ciseness of propositional logic. A Boolean game consists of a set of 
players, each of whom controis a set of propositional variables and 
has a specific goal expressed by a propositional formula. We show 
here that Boolean games are a very simple setting, yet sophisticated 
enough, for studying coalitions. Due to the fact that players have 
dichotomous preferences, the following notion emerges naturally: a 
coalition in a Boolean game is efficient if it guarantees that the goal 
of each member of the coalition is satisfied. We study the proper- 
ties of efficient coalitions, and we give a characterization of efficient 
coalitions. 


1 Introduction 


Boolean games (Harrenstein et al., 2001; Harrenstein, 2004; Dunne and 
van der Hoek, 2004; Bonzon et al., 2006b) are a logical setting for repre- 
senting strategie games in a succinct way, taking advantage of the expressive 
power and conciseness of propositional logic. Informally, a Boolean game 
consists of a set of players, each of whom controls a set of propositional 
variables and has a specific goal expressed by a propositional formula’. 
Thus, a player in a Boolean game has a dichotomous preferenee relation: 
either her goal is satisfied or it is not. This restrietion may appeal’ at first 
glance unreasonable. However, many concrete situations can be modelied as 
games where agents have dichotomous preferences (we give such an exam- 
ple in the paper). iVIoreover, due to the fact that players have dichotomous 
preferences, the foliowing simple (yet sophisticated enough) notion emerges 


1 We refer here to the version of Boolean games defined in (Bonzon et al., 2006b), that 
generalizes the initial proposal by Harrenstein et al. (2001). 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 283-297. 
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naturally: a coalition in a Boolean game is efficient if it guarantees that all 
goals of the members of the coalition are satisfied. Our aim in the following 
is to define and characterize efficient coalitions, and see how they are related 
to the well-known concept of core. 

Aftel' recalling some background of Boolean games in Section 2, we study 
in Section 3 the properties of effectivity functions associated with Boolean 
games. In Section 4 we study in detail the notion of efficient coalitions. We 
give an exact characterization of sets of coalitions that can be obtained as 
the set of efficient coalitions associated with a Boolean game, and we relate 
coalition efficiency to the notion of core. Related work and further issues 
are discussed in Section 5. 


2 n-player Boolean games 


For any finite set V = fa, b,...} of propositional variables, Ly denotes the 
propositional language built up from V, the Boolean constants Tand L, 
and the usual connectives. Formulas of Ly are denoted by y,~# etc. A 
literal is a variabie x of VOI the negation of a literal. A term is a consistent 
conjunction of literals. A clause is a disjunction of literals. I[ y E Ly, 
then Var(y) (resp. Lit(a)) denotes the set of propositional variables (resp. 
literals) appearing in y. 

2” is the set of the interpretations for V, with the usual convention that 
for M E 2V and x E V, M gives the value irue to x if x E M and false 
otherwise. |— denotes the consequence relation of classical propositional 
logic. Let VI C V. A VlI-interpretation is a truth assignment to each 
variabie of V/, that is, an element of gv" VI-interpretations are denoted 
by listing all variables of VI, with a - symbol when the variabie is set to 
false: for instance, let VI = fa, b,d}, then the VI-interpretation M = fa, d} 
assigning a and d to true and b to false is denoted by abd. I[ Var(y) C X, 
then M odx (p) represents the set of X -interpretations satisfying y. 

IL {Vi,..., Vp} is a partition of V and {MI'...'Mp} are partial in- 
terpretations, where M, E 2”, (Mj,,...'M,) denotes the interpretation 
M,U...UM,. 

Given a set of propositional variables V, a Boolean game on V is an n- 


player game", where the actions available to each player consist in assigning 
a truth value to each variabie in a given subset of V. The preferences of 
each player i are represented by a propositional formula p; formed using 
the variables in V. 


Definition 2.1. An n-player Boolean game is a 5-tuple (N, Vz, f, ®), 
where 


2 In the original proposal (Harrenstein et al., 2001), Boolean games are two-players 
zero-sum games. However the model ean easily be generalized to n players and non 
necessarily zero-sum games (Bonzon et al., 2006b). 
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e N= {I, 2,...,n} is a set of players (also called agents); 
e V is a set of propositional variables; 
e tt: V — N isa control assignment function; 


ef= {yi ... ,/n/ is a set of constraints, where each y; is a satisfiable 
propositional formula of Lr(i); 


e P= {y1, ....Yn} isa set of goals, where each g; is a satisfiable formula 
of Ly. 


A 4-tuple (N, V,7,T), with N, V,7,f defined as above, is called a pre- 
Boolean game. 


The control assignment function st maps each variabie to the player who 
controls it. For ease of notation, the set of all the variables controlled by 
a player i is written m; such as m; = {X E V | a(x) = i}. Each variabie is 
controlled by one and only one agent, that is, {71, --- , nn } forms a partition 
of V. 

For each i, y; is a constraint restricting the possible strategy profiles for 
player i. 


Definition 2.2. Let G = (N, V,tt, f, ®) be a Boolean game. A strategy" 
for player i in G is a Ifi-interpretation satisfying 7i. The set of strategies 
for player i in G is si = ¢si E 2% lsi H yi}. A strategy profile s for Gis 
a n-tuple s (SL, S2,..., sn) Where for all i, si E si. S = SIX... X Sn is 
the set of all strategy profiles. 


Note that since {71, ...,7,} forms a partition of V, a strategy profile s 
is an interpretation for V, i.e, sE 2”. The following notations are usual in 
game theory. Let S= (SJ, ..., sn) be a strategy profile. For any nonempty 


set of players J C N, the projection of s on J is defined by S7 = (Si)iEI and 
s_p = snv' If! = {i}, we denote the projection of s on {i} by si instead 
of S{i}; similarly, we note S-i instead of s-/i/' my denotes the set of the 
variables controlled by J, and #-7 = 7y\;. The set of strategies for Z C N 
is sz = XiEISi, and the set of goals for I C N is r fees Pi. 

If s and si are two strategy profiles, (s_z, s4) denotes the strategy profile 
obtained from s by replacing si with s/ for all i E Z. 

The goal y; of player i is a compact representation of a dichotomous 
preference relation, or equivalently, of a binary utility function wù; :S > 
{O, I} defined by w;(s) = 0 if s H ~y; and u;(s) = 1 if s H yi. s is at least 
as good as si for i, denoted by s >, si, if ui(s) > w:(s’), or equivalently, 


3 In this paper, only pure strategies are considered. 


286 E. Bonzon, M.-C. Lagasquie-Schiex, J. Lang 


if s Æ my; implies si =| ay,; S is strictly better than si for i, denoted by 
s >, 8’, if uils) > wz(s’), or, equivalently, s = y; and si = nyi. 

Note that this choice of binary utilities clearly implies a loss of generality. 
However, some interesting problems, as in Example 4.3, have preferences 
that are naturally dichotomous, and Boolean games allow to represent these 
problems in a compact way. Furthermore, Boolean games can easily be 
extended so as to allow for non-dichotomous preferences, represented in 
some compact language for preferenee representation (see Bonzon et al., 
2006a). 


3 Coalitions and effectivity functions In Boolean 
games 


Effectivity functions have been developed in social choice to model the abil- 
ity of coalitions (Moulin, 1983; Abdou and Keiding, 1991; Pauly, 2001). As 
usual, a coalition Cis any subset of N. Nis called the grand coalition. 
Given a set of alternatives S$ from which a set of agents N have to choose, 
an effectivity function Eff: 2% 4 22° associates a set of subsets of S with 
each coalition. X E Eff(C) is interpreted as "coalition C is effective for X". 


Definition 3.1. A coalitional effectivity function is a function 
Eff: 2N — 27° which is monotonie: for every coalition C C N, X E Eff(C) 
implies Y E Eff(C) whenever X < Y < S. 


The function Eff associates to every group of players the set of outcomes 
for which the group is effective. We usually interpret X E Eff(C) as "the 
players in C have a joint strategy for bringing about an outcome in X”. 

In (Pauly, 2001), the meaning of "effective" is precised in the framework 
of strategie games by defining "a-effectivity": a coalition C C N is a- 
effective for X C S if and only if players in C have a joint strategy to 
achieve an outcome in X na matter- what strategies the other players choose. 

As Boolean games are a specific case of strategie games, we would like 
to define a-effectivity functions in this framework. One of the features of 
Boolean games is the definition of individual strategies as truth assignments 
of a given set of propositional variables. We might wonder how restrictive 
this specificity is. In this section we study Boolean games from the point 
of view of effectivity functions. Clearly, the definition of Si as M odz; (fi) 
induces some constraints on the power of players. Our aim is to give an 
exact characterization of a-effectivity functions induced by Boolean games. 
Since in Boolean games the power of an agent i is her goal y;, it suffices to 
consider pre-Boolean garnes only when dealing wit.h effectivity Iunctions. A 
pre-Boolean game Ginduces an a-effectivity function Effe as follows: 


Definition 3.2. Let G = (N, V,rtt, I") be a pre-Boolean game. The coali- 
tional a-effectivity function induced by G is the function Effe: 2% > 2? 
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defined by: for any X C Sand any C CN, X E Effe(C) if there exists 
Sc E Sc sueh that for any s_c E S-C, (Sc, s-c) E x.4 


This definition is a partieular ease of the a-effeetivity function indueed 
by a strategie game (see Pauly, 2001, Chapter 2). Therefore, these functions 
satisfy the following properties (cf. Pauly, 2001, Theorem 2.27): (i) YC C N, 
o ¢ Eff(C); (ii) YC < N, S E Eff(C); Gii) for all X < S, if X ¢ Eff(0) 
then X E Eff(N); (iv) Eff is superadditive, that is, if for all C, Cf C N 
and X, Y C S, X E Eff(C) and Y E Eff(C?: then X NY E Eff(C U Cf). 
An effeetivity function satisfying these four properties is ealled strongly 
playable. Note that strong playability implies regularity and eoalition- 
monotonieity (Pauly, 2001, Lemma 2.26). 

However, pre-Boolean games are a speeifie ease of strategie game forms, 
therefore we would like to have an exaet eharaeterization of those effeetivity 
functions that correspond to a pre-Boolean game. We first have to define 
two additional properties. Define At(C) as the minimal sets in Eff( C), that 
is, At(C) = {X E Eff(C) | there is no Y E Eff(C) sueh that Y C X}. 


Atomicity: Eff satisfies atomicity if for every C C N, At(C) forms a par- 
tition of S. 


Decomposability: Eff satisfies decomposability if for every I J C N and 
for every X C S, X E Eff(IU J) if and only if there exist Y E Eff(I) 
and ZE Eff(J) sueh that X = YNZ. 


Note that deeomposability is a strong property that implies superadditivity. 


Proposition 3.3. A eoalitional effeetivity function Eff satisfies (1) strong 
playability, (2) atomieity, (3) deeomposability and (4) Eff(N) = 25\ 0 if 
and only if there exists a pre-Boolean game G = (N, V,rt, T) and an injeetive 
function u : S — 2” sueh that for every C C N: Effe(C) = {u(X) 1X E 
Eff(Cn. 


Sketch of PTOOf5 The (<=) direetion does not present any diffieulty: we ean 
easily prove than Effe satisfies strong playability (from Pauly, 2001, The- 
orem 2.27), atomieity, deeomposability and Effe(N) = 25\ O. As p isa 
bijeetion between S and y(S), these properties transfer to Eff. 

For the (=>) direetion, we first show that for every s E S, there exists a 
unique (Zl, ...,Zn) sueh that for every i, Z; E At(i) and Z/N...nZn = {s}. 
Then, we build G from Eff as follows: 


4 Note that elfectivity functions induced by pre-Boolean games may be equivalently 
expressed as mappings Elf : 2N — 2LV from coalitions to sets of logical formulas: yg E 
EIf() if Modg , (p) E EIf(1). This definition obviously implies syntax-independence, 
that is, if p = % then YE Elf(I) iff Ww E Elf(/). 

5 A complete version of this proof can be found in (Bonzon et al., 2007). 
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e for every i, number Af(i): let r; be a bijective mapping from At(i) to 
{O, 1,..., ZA{(i)] - I}. Then create Pi = [log,|At(z)|] propositional 
variables z},... ,v?*. Finally, let V = {aj IiE N1<j < pj}; 


e foreach i: m; = {a},... 22°}; 

e for each i and each j < Pi, let €; be the jth digit in the binary 
representation of Pi. Note that Eip; = 1 by definition of Pi. If x is a 
propositional variabie then we use the following notation: O.X= ax 
and Lx = x. Then define 


“i = No (AN ega +27) 


jE{2, .. pi} ei5=0 1<k<j—1 


e finally, for each s E S, let p(s) E 2” defined by: 2} E p(s) if and only 
if the jth digit of the binary representation of r;(Zi (s)) is 1. 


For every i E N and every Z E At(i), let k = Ti(Z) and Si(Z) the strategy 
of player in i in G corresponding to the binary representation of k using 
{Xil, ... xt}, xil being the most significant bit. For instance, ifpi = 3 
and Ti(Zi) = 6 then Si(Z) = (xiỌl, Xi2, 'Xi3). Q.E.D. 


Note. To follow the proof, it may be helpful to see how this construction 
works on an example. Let N = {I, 2,3}, S = {1, 2,3,4, 5, 6,7,8,9, A, B, C), 
At(l) = {1234, 5678, 9ABC}, At(2) = {13579B,2468AC}, At(3) = 
{12569C, 3478AB} (parentheses for subsets of S are omitted-1234 means 
{I, 2,3,4} and so on). By decomposability, we have At(12) = {13, 24, 57, 68, 
9B,AC}, At(13) = {12,34,56,78,9C,AB}, and At(23) = {159,37B, 
26C, 48A}. IAt(I)I = 3, therefore p: = 2. JAt(2)1 = JAt(3)1 = 2, there- 
fore P2 = P3 = 1. Thus, V = {x 11,x 12, x21, x31}. Let At(l) = {Zo, Zl, Z2}, 
that is, 7,(1234) = 0, rı(5678) = 1 and rı(9ABC) = 2. Likewise, 
T2(13579B) = 0, T2(2468AC) = 1, T3(12569C) = 0 and 73(3478AB) = 1. 
Consider s = 6. We have s = 5678 N 2468AC N 12569C, therefore sc = 
p(s) = (7011,212,291,-4%31). The constraints are yı = (sıl — ,x/]2) 
Yara: => Te, 

Then, we show that for every C, Effe(C) = w(Eff(C)). The proof, 
though rather long, does not present any particular difficulty. See (Bonzon 
et al., 2007). 


’ 


4 Efficient coalitions 
4.1 Definitions and characterization 


We now consider full Boolean games and define efficient coalitions. Infor- 
mally, a coalition is efficient in a Boolean game if and only if it has the 
ability to jointly satisfy the goals of all members of the coalition: 
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Definition 4.1. Let G = (N, V,x,IT,®) be a Boolean game. A coalition 
C CN is efficient if and only if Isc E Sc such that Vs_c, Se H Nicc Yi: 
The set of all efficient coaJitions of a game G is denoted by EC(G). 


Example 4.2. Let G = (N, V,T,7,®) where V = {a,b,c}, N = {1,2,3}, 
yi = T for every i, mı = fa}, m2 = {b}, m3 = {c}, pı = (~a A b), p2 = 
(~a V =c) and p3 = (=A 'c). 

Observe first that Yı A 3 is inconsistent, therefore no coaJition con- 
taining {1,3} can be efficient. {1} is not efficient, because y; cannot be 
made true only by fixing the value of a; similarly, {2} and {3} are not effi- 
cient either. {1,2} is efficient, because the joint strategy S/1,2} = ab is such 
that S/1,2) H yi A p2. {2,3} is efficient, because $/2,3} = be EK p3 A ys. 
Obviously, 0 is efficient", because yg Pees yi = T is always satisfied. 
Therefore, EC(G) = {0, {1,2}, {2, 3}}. 


From this simple example we see already that EC is neither downward 
closed nor upward closed, that is, if C is efficient, then a subset or a superset 
of C may not be efficient. We also see that EC is not closed under union 
or intersection: {1,2} and {2,3} are efficient, but neither {1,2}N {2,3} nor 
{1,2}U{2,3} is. 


Example 4.3 (Kidney exchange, aftel' Abraham et al., 2007). Consider n 
pairs of individuals, each consisting of a recipient R; in urgent need of a kid- 
ney transplant, and a donor Di who is ready to give one of her kidneys to 
save Ri. As D;’s donor kidney is not necessarily compatible with Ri, a strat- 
egy for saving more people consists in considering the graph ({1,... , n}, E) 
containing a node i E 1,...,n for each pair (Di, R;) and containing the edge 
(i,j) whenever D,’s kidney is compatible with Rj. A solution is any set of 
nodes that can be partitioned into disjoint cycles in the graph: in a solution, 
a donor Di gives a kidney if and only if R; is given one. An optimal solution 
(saving a maximum number of Jives) is a solution with a maximum number 
of nodes. The problem can be seen as the following Boolean game G: 


«e N=(//,...,n}; 


e V = fgij i,j E {1,...,n}}; gij being true means that Di gives a 
kidney to Rj. 


e Tti = {gij; 1 Sj <n}; 


e for every i, Yi = Nite ,(gij N gik) expresses that a donor cannot give 
more than one kidney. 


6 One may argue this makes little sense to say that the empty coalition is efficient. 
Anyway, the definition of an efficient coalition could be changed so as to exclude O, 
further notions and results would be unchanged. 
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e for every i, Yi = VG.0EE gji expresses that the goal of i is to be given 
a kidney that is compatible with Ri. 


For example, take n = 5and E = {(1, 1), C1, 2), (2,3), (2,4), (2,5), (3, 1), 
(4,2), (5,4)}. Then G = (N, V,I, rt, ®), with 


e JV = {1,2,3,4,5} 

e V- feij I< iy <5}; 

e Vi, Yi = Njzk (gij M gik) 

+- Tı = {gll, g12, g13, g14,g15}, and similarly for 7, etc. 

© Y1 = g11 V 931; 2 = g12 V 942; P3 = 923; P4 = Goa V 954; P5 = 925. 


The corresponding graph is depicted below. 


Clearly enough, efficient coalitions correspond to solutions. In our ex- 
ample, the efficient coalitions are O, {I}, {2,4}, {1,2,4}, {I, 2,3}, {2,4,5} 
and {1,2,4,5}. 


We have seen that the set of efficient coalitions associated with a Boolean 
game may not be downward closed nor upward closed, nor closed under 
union or non-empty intersection. We find that it is possible to characterize 
the efficient coalitions of a Boolean game. 


Proposition 4.4. Let N = {I,...,n} be a set of agents and SC E oe a 
set of coalitions. There exists a Boolean game G over N such that the set of 
efficient coalitions for G is SC (i.e. EC(G) = SC) if and only if SC satisfies 
these two properties: 


(1) 0 E SC. 
(2) for all Z JE SC such that Zn J =0, I UJE SC. 


Thus, a set of coalitions corresponds to the set of efficient coalitions for 
some Boolean game if and only if (a) it contains the empty set and (b) it is 
closed by union of disjoint coalitions. 
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Sketch of proo],' The (=>) direetion is proven easily; intuitively, when two 
disjoint eoalitions / and Jare efficient, eaeh one has a strategy guaranteeing 
its goals to be satisfied, and the joint strategies of / and J guarantee that 
the goals of all agents in / U Jare satisfied. As seen in Example 4.2, this 
is no longer true when / and Jinterseet. The (<) direetion of the proof is 
more involved and needs the following Boolean game G to be eonstrueted 
for eaeh set of coalitions SC satisfying (1) and (2): 


e V= {connect(i, j) li, JEN} (all possible eonneetions between play- 
ers ); 


R vi, Yi = T; 


’ 


» Ti = {connect(i,j) Ij E N} (all eonneetions from player i); 


e. Pig Vrese ix Fi, where 


F,= ( A connect(j, k)) N ( A -,connect(j, k)) 


j,kEI jELkrfJ 


(player i wants all the players of her eoalition to be eonneeted with 
eaeh other and diseonneeted from the players outside the eoalition). 


We want to show that ECg = SC (where ECg is the set of effieient 
eoalitions for G). We first show that SC C ECg. Let / E SC. If every agent 
iE / plays (Njer connect(i, j)) A (Negr -,connect(i, k)), then Y; is satisfied 
for every i E Z. Henee, J is an effieient coalition for G and SC is included 
in EC(G). 


In order to prove that ECg C SC, we define a coueruig ofa coalition | 
by disjoint subsets of SC as a tuple Ce (C; li E Z) of coalitions sueh that: 
(a) for every KET, Ci; E SC; (b) for all Cj, Ci; E C, either Cj = Ci; or 
C; N Ci; = O; (e) for every i E Z, i E Ci. Let Cov(SC, 1) be the set of all 
eovering of / by disjoint subsets of SC. 

For instanee, if SC {0, 1,24,123, 124} then Cov(SC, 12) 
{(I, 24), (123, 123), (124,124) }8, Cov(SC, 124) = {(I, 1,24), (1,24,24), 
(124,124,124)}, Cov(SC, 123) = {(123, 123, 123)} and Cov(SC, 234) = 
Cov(SC, 1234) = O. 


The proof goes along the following steps: 
LI For any eollection Col = {Ci,i = 1,...,q} © 2", Ni<i<g Fc, is sat- 


isfiable if and only if for any i,j E {I,...,gj, either C; = Cj or 
C, nG O. 


7 A complete version of this proof can be found in (Bonzon et al., 2007). 
8 There are two players in Z = {I, 2}, therefore each G in Cov(SC, 12) contains 2 coali- 
tions, one for each player, satisfying (a), (b) and (c). 
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L2 From LI, we deduee that VJ Æ 0, ®; is equivalent to 


V I\ Fei. 


êECov(SC,I) iet 


L3 From property (2) (assumption of Proposition 4.4) and L2, we ean prove 
that if / C 2", then ®y is satisfiable if and only if there exists J E SC 
sueh that | < J. 


Let J be an effieient coalition sueh that J ¢ SC (whieh implies J Æ O, 
beeause by assumption 0 E SC). 


e If Z= N then there is no J E SC sueh that J C J (beeause I ¢ SC), 
and then L3 implies that ®; is unsatisfiable, therefore / eannot be 
effieient for G. 


e Assume now that J Æ N and define the following I-strategy Sr (I = 
NNI): for everyi E Í, si = {,connect(i,j) Ij E I} (plus whatever 
on the variables connect(i,j) sueh that j ¢ J). Let Ge (Cii EI) E 
Cov(SC,1). 

We first claim that there is a i* E Z sueh that Cj is not eontained in 
I. Indeed, suppose that for every i E Z, C, C I. Then, beeause i E C, 
holds for every i, we have UiEI C, = I Now, C, E SC for all i, and 
any two distinet Ci, Cj are disjoint, therefore, by property (2) we get 
|l E SC, whieh by assumption is false. 

Nuw, let k E Ci» \ I (such a k exists beeause Ci» is nut euntained 
in Z). Now, the satisfaetion of FG requires connect(k, i*) to be true, 
beeause both i and k are in Ci. Therefore sk |= , Foj, and a fortiori 
«t | ,Fci, whieh entails «t = 'AiEIFCj. 


This being true for any CŒ E Cov(SC,I), it follows that we have 
«t F ÀêECov(SC,I) = Nier FG that is, «t F , V &ECow(SC,) Nier Fei: 
Together with L2, this entails «t |: ~@y. Hence, / does not control 
P; and / eannot be effieient for G. 


Q.E.D. 


The notion of effieient coalition is the same as the notion of sueeessful 
eoalition in qualitative eoalitional games (QCG) introdueed in (Wooldridge 
and Dunne, 2004), even if, as we diseuss in Seetion 5, QCG and Boolean 
games are quite different. 
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4.2 Efficient coalitions and the core 


We now relate the notion of efficient coalition to the usual notion of core of 
a coalitional game. In coalitional games with ordinal preferences, the core 
is usually defined as follows (see e.g., Aumann, 1967; Owen, 1982; Myerson, 
1991): a strategy profile s is in the core of a coalitional game if and only if 
there exists no coalition C with a joint strategy Sc that guarantees that all 
members of Care better off than with s. Here we consider also a stronger 
notion of core: a strategy profile s is in the strong core of a coalitional 
game if and only if there exists no coalition C with a joint strategy Sc that 
guarantees that all members of C are at least as satisfied as with s, and at 
least one mernber of C is strictly better off than with s. 


Definition 4.5. Let C be a Boolean game. The (weak) core of C, denoted 
by WCore(C), is the set of strategy profiles s = (s1,..., sn) such that there 
exists no @@MN and no Sc E Sc such that for every i E C and every 
s_c E S-c, (sc, s_c) >i 8. 


The strong core of a Boolean game C, denoted by SCore(C), is the set of 
strategy profiles s = (Sl, ...,sn) such that there exists no @@N and no 
Sc E Sc such that for every i E C and every s_c E S-c, (sc, s-c) =; S 
and there is an i E C such that for every S-c E S-c, (sc, 8_c) >i s. 


This concept of weak core is equivalent? to the notion of strong Nash 
equilibrium introduced by Aumann (1959), where coalitions form in order 
to correlate the strategies of their members. This notion involves, at least 
implicitly, the assumption that cooperation necessarily requires that players 
be able to sign "binding agreements": players have to follow the strategies 
they have agreed upon, even if some of them, in turn, might profit by 
deviating. However, if players of a coalition C agreed for a strategy Sc, at 
least one player i E C is satisfied by this strategy: we have Ji E C such 
that s F Yi. 


The relationship between the (weak) core of a Boolean game and its set 
of efficient coalitions is expressed by the following simple result. The proofs 
of following results can be found in (Bonzon et al., 2007): 


Proposition 4.6. Let C = (N, V,I, #,P) be a Boolean game. Then s E 
WCore(C) if and only if s satisfies at least one member of every efficient 
coalition, that is, for every C E EC(C), s = Viec Pi. 


In particular, when no coalition of a Boolean game C is efficient, then 
all strategy profiles are in WCore( C). Moreover, the weak core of a Boolean 
game cannot be empty: 


9 This equivalence is easily shown: it is just a rewriting of the definition given in (Au- 
mann, 1959). 
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Proposition 4.7. For any Boolean game G, WCore(G) Æ 0. 


The strong core of a Boolean game is harder to characterize in terms of 
efficient coalitions. We only have the fol!owing implication. 


Proposition 4.8. Let G = (N, V,I', ,@) be a Boolean game, and s be a 
strategy profile. If s E SCore(G) then for every C E EC(G) and every i E C, 


SF Pi- 


Thus, a strategy in the strong core of G satisfies the goal of every mernber 
of every efficient coalition. The fol!owing counterexample shows that the 
converse does not hold. 


Example 4.9. Let G = (N, V,I, tt, ®) be a Boolean game. We have: 


V = {abcde f}, N = {I,2,3,4,5,6}, y = T for every i, Tı = fa}, 
T2 = {b}, T3 = {c}, T4 = {d}, T5 = fe}, Te = {f}, Yi = bVd, Y2 = aVe, 
g3 = ,b Vd, pa=e, ys =naN,b\ ,c and y= ,al,cf <d. 


This game has two efficient coalitions: {1,2} and {2,3}. 


Let s = abcdef. We have s H Yin v2 A v3 Amp, A mys A mys. So, 
YC E EC(G), Vie C, s E yi. 
However, s ¢ SCore(G): IC” = {I, 2,3,4,5} C N such that dsc = abcde 


pı n p2 A p3 A pa AN 75. So, Ys-c, (sc, s-c) z1 s (sc, s-c) 52 S, 
(sc, s-c) &3 s, (sc, s-c) &s s, and (sc, s-c) >4 s. s € SCore(G). 


Note that the strong core of a Boolean game can be empty: in Exam- 
ple 4.2, the set of efficient coalitions is {0, {I, 2}, {2,3}}, therefore there 
is no s E S such that for al! C E EC(G), for al! i E C, s = y, therefore, 
SCore(G) = O. However, we can show than the non-emptiness ofthe strong 
core is equivalent to the following simple condition on efficient coalitions. 


Proposition 4.10. Let G = (N, V,I, t,®) be a Boolean game. We have 
the following: 

Score(G) #0 if and only if U{C C N ICE EC(G)} E EC(G), that is, 
if and only if the union of al! efficient coalitions is efficient. 


5 Conclusion 


We have shown that Boolean games can be used as a compact representation 
setting for coalitional games where players have dichotomous preferences. 
This specificity led us to define an interesting notion of efficient coalitions. 
We have given an exact characterization of sets of coalitions that correspond 
to the set of efficient coalitions for a Boolean game, and we have given several 
results concerning the computation of efficient coalitions. 
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Note that some of our notions and results do not explicitly rely on the 
use of propositional logic. For instance, efficient coalitions can be defined 
in a more general setting where goals are simply expressed as nonempty 
sets of states. However, many notions (in particular, the control assignment 
function 7) become much less clear when abstracting from the propositional 
representation. 

Clearly, a limitation of our results is that they apply to dichotomous 
preferences only. However, as illustrated on Example 4.3, some problems 
are naturally expressed with dichotomous goals. Moreover, it is always 
worth starting by studying simple cases, especially when they already raise 
complex notions-?. 

As Boolean games, qualitative coalitional games (QCG), introduced in 
(Wooldridge and Dunne, 2004), are games in which agents are not assigned 
utility values over outcomes, but are satisfied if their goals are achieved. A 
first difference between QCG and Boolean games is that there is no control 
assignment function in QCG. A second one is that each agent in QCG can 
have a set of goals, and is satisfied if at least one of her goals is satisfied, 
whereas each agent in Boolean games has a unique goal. However, QCG's 
characteristic function, which associates to each coalition C the sets of goals 
that members of C can achieve, corresponds in Boolean games to the set 
W(C) = {X C{y1,...,Gn} such that Isc E Se : Sc Fy}. 

Coalition logic (Pauly, 2001) allows to express, for any coalition C and 
any formula ọ, the ability of C to ensure that y hold (which is written [C]y). 
In Boolean games, the power of agents, expressed by the control assignment 


function zz, is still in the metalanguage. Expressing ¢t within coalition logic 
would however be possible, probably using ideas Irorn (van der Hoek and 
Wooldridge, 2005). The next step would then consist in introducing goals 
into coalition logic. This is something we plan to do in the near future. 
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Abstract 


According to the optimal assertions approach of Benz and van Rooij 
(2007), conversational implicatures can be calculated based on the as- 
sumption that a given signal was optimal, i.e. that it was the sender's 
best choice if she assumes, purely hypothetically, a particular naive 
receiver interpretation behavior. This paper embeds the optimal as- 
sertions approach in a general signaling game setting and derives the 
notion of an optimal signal via a series of iterated best responses (cf. 
Jager, 2007). Subsequently, we wil! compare three different ways of 
interpreting such optimal signals. It turns out that under a natural 
assumption of expressibility (i) the optimal assertions approach, (ii) 
iterated best response and (iii) strong bidirectional optimality theory 
(Blutner, 1998, 2000) all prove equivalent. We then proceed to show 
that, if we take the iterated best response sequence one step further, 
we can account for M-implicatures (Horn's division of pragmatic la- 
bor) standardly in terms of signaling games. 


Often we express more with the use of our words than what those words 
mean literally. For example, if you were to say that this observation is 
not particularly new, I would clearly get the hint and understand that you 
meant to say that it is more than just not particularly new, indeed a werking 
standard in linguistic pragmatics. Such conoersationol implicaiures were 
first studied by Grice (1989) and stil! concern the community in various 
ways. In particular, recent years saw an increasing interest in game-theoretic 
models of conversational implicature calculation, and this study belongs to 
this line of research. It provides a formal comparison of selected previous 
approaches which extends to a uniform synchronie account of different kinds 
of conversational implicatures. 

The paper is organized as fol!ows. Section 1 briefiy reviews the classifica- 
tion of conversational implicatures into 1-, Q- and M-implicatures. Section 2 
introduces a game-theoretical model of implicature calculation: a signaling 
game with exogenously meaningful signais. We wil! see in Section 2.3 that 


Krzysztof R. Apt, Robert van Rooij (eds.). New Perspectives on Games and Interaction. 
Texts in Logic and Games 4, Amsterdam University Press 2008, pp. 299-312. 
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the standard solution concept for signaling games is not strong enough to 
account for the empirical observations. The optimal asserteems approach. of 
Benz and van Rooij (2007), which is introduced in Section 3.1, is an at- 
tempt to solve this problem. According to the optimal assertions approach, 
conversational implicatures can be calculated based on the assumption that 
a given signal was optima!. Sections 3.2 and 3.3 then compare three ways 
of interpreting such optimal signais: (i) the pragmatic interpretation rule 
of Benz and van Rooij (2007), (ii) iterated best response and (ili) strong 
bidirectional optimality theory (Blutner, 1998, 2000). It turns out that if 
we assume a sufficiently expressible stock of possible signais, all three ap- 
proaches prove equivalent. However, it also turns out that M-implicatures 
(Horn's division of pragmatic labor) cannot be accounted for based solely 
on the assumption that the received form was optima!. We will conclude 
that some aid from the refinement literature, in particular Cho's and Kreps’ 
(1987) intuitive criterion, is necessary and sufficient to account uniformly 
for all 1-, Q- and M-implicatures. 


1 Kinds of conversational implicatures 


Neo-Gricean pragmatics (Atlas and Levinson, 1981; Horn, 1984) distin- 
guishes J-implicatures (1) and Q-implicatures (2). 


(1) John has a very efficient secretary. 
~ John has a very efficient female secretary. 


(2) John invited some of his friends. 
~» John did not invite all of his friends. 


I-implicatures like (1) are inferences to a stereotype: the sentence is asso- 
ciated with the most likely situation consistent with its semantic meaning. 
Q-implicatures like (2), also called scalar implicatures, are a strengthening 
of the literal meaning due to the presence of more informative alternatives 
that were not used: since the speaker only said that some of John's friends 
were invited, we infer that the compatible stronger claim that all of John's 
friends were invited-a claim that we may assume relevant if true---does 
not hold, for otherwise the speaker would have said so-as she is assumed 
cooperative and informed. 

A third kind of implicature, called M-implicature by Levinson (2000), is 
given in (3). 


(3) The corners of Sue's lips turned slightly upwards. 
~» Sue didn't smile genuinely, but faked a smile. 


In (3) we naturally infer that something about the way Sue smiled was ab- 
normal, non-stereotypical or non-standard, because the speaker used a long 
and complicated form where she could have used the simple expression (4). 
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(4) Sue smiled. 


M-implicatures were also discussed by Hom (1984) and have been addressed 
as Horn's division of pmgmatic labor thereafter. It has become customary 
to assume that both sentences (3) and (4) are semantically equivalent, but, 
when put to use, the longer form (3) gets to be associated with the non- 
stereotypical situation, while the short form (4) gets to be associated with 
the stereotypical situation. 


2 Implicatures via signaling games 
2.1 Interpretation frames 


A fairly manageable set of contextual parameters plays a role in the neo- 
Gricean ciassification of implicatures: we distinguish various meanings that 
are more or less stereotypical and we compare different forms with respect 
to their semantic meaning and complexity. We can then capture any such 
configuration of contextual parameters that are relevant for the computation 
of implicatures in an inierpretation frame. 


Definition 2.1 (Interpretation Frame). An interpretation frame is a tuple 
F = (W, P, F,c, LID 


where W is a finite set of worlds or situations, P is a probability distribution 
over W with the usual properties,1 F is a set of forms or signals which the 
sender may send, c : F — R is a cost function and [|] : F = A(W) isa 
semantic denotation function mapping forms to subsets of W. 


We assume for convenience that P(w) +Æ 0 for all worlds w E W. We 
would also like to rule out certain rather pathological situations where there 
are worlds which simply cannot be expressed by any conventional signal: 


Assumption 2.2 (Semantic Expressibility). We only consider interpreta- 
tion frames in which all worlds are semantically expressible: for all worlds 
w there has to be a form f such that w E [fl]. 


The kinds of implicatures described in the previous section correspond 
to abstract interpretation frames as follows: 


e The I-frame is an interpretation frame Fr = (W, P, F,c,[]]) where 
W = {w,v}, P(w) > P(v) £0, F = {j,g,h}, cU) < c(g),c(h) and 
[fll = w, [g] = {v} and [hl] = {w}. The observed J-implicature play 
is to interpret f as wand to send f in w only. 


1 P(w) E [0,1], for all w E W; P(A) = XY wga P(w), for all AC W; P(W) = 1. 
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e The Q-fmme is an interpretation frame Fa = (W,P,F,c, [.]) where 
W = {w,v}, P(w) > P(v) £0, F = {j,g}, c(j) = c(g) and [fll = W, 
[g] = {v}. The observed Q-implicature play is to interpret f as wand 
to send f in w only. 


e The /vl-fmme is an interpretation frame Fn = (W, P, F, c, [.]]) where 
W = {w,v}, P(w) > P(v) £0, F = fj,g}, c(j) < c(g) and [1] = 
[gli = W. The observed Ivl-implicature play is to interpret f as wand 
to send f in w only, as wel! as to interpret 9 as v and to send 9 in v 
only. 


2.2 Interpretation games 


Interpretation frames capture the relevant aspects of the situation in which 
communication takes place. The communication itself can best be imagined 
as a signaling game: nature selects a world w E W-cal! it the actual world 
in a given play-with probability P(w) and reveals it to the sender who 
in turn chooses a form f E F. The receiver does not observe the actual 
world, but observes the signal f. He then chooses an action A. Sender and 
receiver receive a payoff based on w, f and A. In the present context, we are 
interested in interpretation games: signaling games in which signals have 
a conventional, compel!ing meaning that the receiver tries to interpret by 
choosing an interpretation action 0 Æ A C W. 


Definition 2.3 (Interpretation Game). An interpretation game is just an 
interpretation frame to which interpretation actions and utilities for sender 
and receiver are added, in other words a tuple 


dof 


G = (F, Act,us, ur) 


where F = (W, P, F, c, [.]]) is an interpretation frame, Act set A(W)\ 0 isa 
set of interpretation actions and uz : F x Actx W — Rare utility functions 
of sender and recelver.é 


+ ifwEA andw E [fl 


def 


url f, A, w) 0 ifw gA andw€ [ffl 
otherwise 
us(f,A, w) “° ur(f, A, w) -C(). 


2 These utilities reftect the mutual desire to communicate which world is actual: the 
more the receiver narrows down a correct guess the better; miscommunication, on the 
other hand, is penalized so that ifthe chosen interpretation does not include the actual 
situation, the payoff is strictly smaller than when it does; a strong penalty is given 
for communication that deviates from the semantic meaning of messages to enforce 
the exogenous meaning of signals. (This last point is objectionable, but it is also not 
strictly necessary. I adept it for ease of exposition since space is limited.) 
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As usual, we identify the receiver's probabilistic beliefs with the probability 
distribution P(-). Costs are assumed nominal: they are small enough to 
make a utility difference for the sender for any two different signals 7 and 
f’ only in case wr(f, A, w) = wrlf’, A, w). 


Definition 2.4 (Strategies). A sender stmtegy is a function o : W —> 
P(F\\ 0 that specifies a set o(w) C F of messages to be sent with equal 
probability when in world w. We call asender strategy o truih-respectsng 
iff for all wand Í whenever Í E o(w) we have WE fiJl. We define also 
o 1(f) “ {we W Ii E o(w)}. Finally, a recesoer stmtegy is a function 
p:F — Act specifying an interpretation for each message. 


Whether an action is preferabie to another depends on what the other 
party is doing. If we fix a strategy for the other party we can define the 
expected utility of each action. 


Definition 2.5 (Expected Utilities). Since the sender knows the actual 
world w, his expected utility of sending the form Å E F given that the 
receiver plays p is actually just his utility in w given Í and the receiver's 
response P(J): 

EUs(J, p,w) = us(f, p(J), w). 
Given that the sender plays a, the receiver's S pened utility of interpreting 
a form 1 for which o™1(J) £0 as A E Act is: 


EUR(A, 0, f) = $2 Plwlo*(f)) x urlf, A, w) 


wEW 


For a truth-respecting sender strategy this simplifies to: 


—1 
BORAS) Ay (2.1) 


If the other party's strategy is given, rationality requires to maximize 
expected utility. A strategy X that maximizes expected utility in all its 
moves given the other party's strategy Y is called a best response to Y. 
For some sender strategies æ and forms 1 it may be the case that several 
actions maximize the receiver's expected utility, and that therefore there is 
no unique best response. Given Equation 2.1, it is easy to see that all (non- 
empty) sets that contain only worlds which are maximally likely according 
to P(-|o~!(J)) are equally good interpretations in expectation:" 


Max acacnEU R(A,o, j) = P(MaxwewP(wlo!(f)))\ o. 


3 We will come back to the question how to interpret messages l in the light of sen der 
strategies g that never use | in Sections 3.2 and 3.4. For the time being, assume that 
EU R (A, ø, I) = 0 is constant for all A if o™1 (I) = O. 

4 We write MaxxEXF(x) der {x EX |3x’ EX: F(x) < F(x')}, for arbit.rary set X 
and function F : X — IR. 
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Assumption 2.6 (Preferred Interpretation). We assume that the receiver 
selects as his best response to a truth-respecting g and f the largest inter- 
pretation action Max,cwP(w|o !(f)). This is because the receiver should 
not discard any possible interpretation without reason; one should not gam- 
bie on proper understanding.f 


The standard solution concept for rational play in a signaling game is a 
perfect Bayesian equilibrium: a pair of strategies that are best responses to 
one another. 


Definition 2.7 (Perfect Bayesian Equilibrium). A pair of strategies (ø, p) 
is a perfect Bayesian equilibrium iff 


(i) for all WE W: o(w) E Max!EFEUs(J, p, w) 
(ii) for all fE F: p(J) E MaxAEActEU R(A, g, J). 


2.3 Pragmatics & the problem of equilibrium selection 


It is easy to verify that 1-,Q- and M-implicature play are all perfect Bayesian 
Equilibria (PBEs) in the corresponding interpretation games, but not uni- 
quely so. Indeed, the straight-forward signaling games approach to impli- 
cature computation faces a problem of equilibrium selection: why is it that 
particular PBEs are observed and not others? 

A natural way of answering this question is to formulate refinements of 
the assumed solution concept. An interesting proposal along these lines is 
given by van Rooij (2008) who observes that the Q-implicature play can be 
singled out as the unique neologism proof PBE (Farrell, 1993) and that the 
M-implicature play can be singled out with the help of Cho's and Kreps’ 
intuitive criterion (Cho and Kreps, 1987). We will piek up this latter idea 
in Section 3.4. Notice, however, that van Rooij's approach deviates from 
a standard signaling game analysis, because in order to arrive at the de- 
sired prediction for the M-frame, van Rooij considers a transition from an 
interpretation frame with just the cheaper message Í, to which at a later 
stage the more costly message 9 is added. The question remains whether 
we cannot account for the observed implicature plays in more conservative 
terms. 


3 Association-optimal signaling 


A recent framework that seeks to give a positive answer to this question is 
Benz and van Rooij's (2007) optimal assertions approach. The basic idea is 
that the receiver may compute implicatures based on the assumption that 
the signal he received was an optimal aseertion. An optimal assertion in 
turn is the best response to a naive, hypothetical interpretation of messages 


5 This assumption replaces the tie-break rule of Benz and van Rooij (2007). 
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that takes into account only the semantic meaning of the message and the 
probabilities of worlds. Benz and van Rooij describe their set-up as a se- 
quence of decision problems: on the hypothesis that the receiver interprets 
signals in a certain, naive way, the sender will choose signals that are opti- 
mal given this receiver strategy and the receiver can then interpret messages 
as optima!. 

Another way of looking at this process is as a sequence of iteraied best 
responses (Cf. Jager, 2007). To point out the connection, I will spell out 
the details of the optimal assertions approach in terms of iterated best 
responses in Section 3.1. I will then, in Section 3.2, show that Benz's and 
van Rooij's interpretation rule deviates slightly from the former iterated best 
response logic in general, but that for a natural subclass of interpretation 
frames-including 1- and Q-frames-the two approaches fall together. In 
Section 3.3, finally, I will conneet both the optimal assertion and the iterated 
best response approach with strong bidirectional optimality theory. 


3.1 Association optimality 


We start with the assumption that the sender says something true: 
oo(w)= UEF Iw E [fi]}. 


We also assume that, given that the sender says something true, the receiver 
will interpret messages as true; in other words, as the sender starts with a 
naive 'truth-only' strategy oo, the receiver maximizes his expected utility 
based on that strategy and plays (as go is truth-respecting): 


po(J) = Maxwew Plwlog ‘(f)) 
= MaxwEWP(wI [fJl)- 


We could think here of a spontaneous, first associative response to the mes- 
sage f: the most likely worlds in which f is true are chosen as the first in- 
terpretation strategy, because these are the worlds that spring to mind first 
when hearing f. We therefore call Po the receiver's association response. 

The association response Po is of course a bad interpretation strategy. In 
fact, it is not a pragmatic interpretation strategy at all, for it leaves out all 
considerations about the interpretation game except |-] and P(-): receipt 
of message f is treated as if it was the observation of the event [fJl. But 
still the association response Po is the rational response to the-admittedly 
non-pragmatic-sender strategy oo. The guiding conviction here is that 
pragmatic reasoning takes semantic meaning as a starting point: if I want 
to know what you meant by a given linguistic sign, I first feed into the 
interpretation machine the conventional meaning of that sign. Therefore, 
as go is a natural beginning, so is the association response P0.6 


6 An anonymous reviewer asks for the difference between Ja.ger's (2007) evolutionary 
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But if this truly is the most reasonable beginning for pragmatic inter- 
pretation, the sender may anticipate the receiver's association response Po 
and choose a best response to it: 


o1(w) = MaX!EFEUs(J, Po, w) 
= {fe FI~Jf E F : EUs(J, Po, w) < EUs(f’, Po, w)} 


Forms in g4 are optimal forms given the receiver's association response. We 
could therefore call them association optimal, or, for short, optimal: a form 
1E F is (association) optimal in a world w iff ; « oj (zw). 

How should the receiver interpret an optimal signal? We'll next consider 
and compare three possible answers to this question. 


3.2 Optimal assertions and iterated best response 


Given semantic expressibility as stated in Assumption 2.2, association op- 
timality is equivalent to Benz's and van Rooij's (2007) notion of an optimal 
assertion. Although the latter notion requires truth of a message for its op- 
timality, it is easy to see that semantic expressibility and optimality entail 
truth. 


Observation 3.1. Given semantic expressibility, 71 is truth-respecting. 


Proo]. Let some Í E F be false in w E W. From semantic express- 
ibility there is a message t' E F which is true in w. But then -1 = 
us(f, po(J), w) <0 < us(f',PO(JI), w), so that L is not association optimal 
in w. Q.E.D. 


If the sender sends an association optimal signal, i.e. if the sender sticks 
to gı, the receiver can again interpret accordingly. Benz and van Rooij 
propose the following interpretation rule based on the assumption that the 
received signal was an Optimal Assertion: pQ4(J) = {w E [ij] IZ is optimal 
in w}. Thich simplifies under Observation 3.1 to 


a) =a (f)- (3.1) 


Notice, however, that this may not be a well-defined receiver strategy in 
our present set-up, for it may be the case that af = O, which is not 
a feasible interpretation action. The same problem also occurs for the best 
response to g1. It is clear what the best response to g; is for messages that 
may be optimal somewhere: if o} '(f) 4 0, we have 


pPR(f) = Maxwew P(wlo; '(f)). (3.2) 


model, which also uses best response dynamics, and the present synchronie approach. 
One obvious difference is that the present model assumes that at each turn a best 
response is selected with probability 1. Another difference is the starting point: in 
Ja.ger's model it is the sender, while in the present model it is the receiver who responds 
first to a strategy that is given by the semantic meaning of the signals. 
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But how should a best response to g} interpret messages that are never 
optimal? Since we defined (tentatively, in Footnote 3) expected utilities 
as constant for all A E Act whenever o!(J) = O, any A E Act is an 
equally good interpretation for a non-optimal f. For our present purpose- 
the comparison of frameworks-it is not important what to choose in this 
case, as long as we choose consistently. We therefore adopt the following 
assumption and refiect on it in Section 3.4 where it plays a crucial role. 


Assumption 3.2 (Uninterpretability Assumption). We assume that the 
receiver resorts to the mere semantic meaning in case a message is uninter- 


pretabie: ifo; (f)= O, then pQA(f) = pRR(f) = [fll 


With this we can show that p?®(f) entails p?4(f) for arbitrary f and 


BR 


interpretation frames. Moreover, ee also entails py, if we assume strorui 


1 


expTessibility: 


Definition 3.3 (Strong Expressibility). An interpretation frame satisfies 
strong expressibility if each world is immediately associated with some mes- 
sage: for each world w there is a form f such that w E po(J). 


Observation 3.4. Under strong expressibility, association optimality im- 
plies inclusion in the association response: if f is association optimal in w, 
then w E po(J). 


Proo]. Assume strong expressibility. If w ¢ po(J), there is a form f’ for 
which w E PO(JI). But then 0 = us(f, po(J), w) < us(f', POJI, w). So f is 


not association optimal in w. Q.E.D. 


Proposition 3.5. For arbitrary interpretation frames it holds that pPR(f) 
(e PENIN: For interpretation frames satisfying strong expressibility it holds 


that pPR(f) = pPA(). 


Proo]. We only have to look at the non-trivial case where a tA #0. Let 
wE prin. Since all worlds have non-zero probabilities we can conclude 
that w Eo, (f). Hence, wE pOA(f). 

Let w E p?(J) and assume strong expressibility. Then w E [f]] and 
f E o;(W). From Observation 3.4 we then know that w E Po(J). That 
means that there is no ui’ for which P(wil [f]]) > P(wl [f]l). But since, by 
Observation 3.1, we know that oi (f) C [f] we also know that there is no 
ui’ for which P(w'|o, (f)) > P(wloy'(f)). Hence w E pB®(f). Q.E.D. 


3.3 Strong bidirectional optimality theory 


A similar conneetion holds with strong Bi-OT (Blutner, 1998,2000). At first 
sight, Bi-OT looks rather different from game-theoretic modeis, because in 
Bi-OT we compare form-meaning pairs (f, w) with respect to a preference 
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order. The idea is that to express a given meaning w with a form J, the 
form-meaning pair (f,W) has to be strongly optima!. Likewise, a form f 
will be associated with meaning W if and only if (f,W) is strongly optima!. 


Definition 3.6 (Strong bidirectional optimality). A form-meaning pair 
(f, W) is strongly optimal iff it satisfies both the Q and the I-principle, 
where: 


(i) (f,w) satisfies the Q-principle iff ~I f’ : J’, w) > (f,w) 
(ii) (f,w) satisfies the I-principle iff dw’ : (f, WI) > (f,w) 


How should we define preference relations against the background of 
an interpretation game? Recall that the Q-principle is asender economy 
principle, while the I-principle is a hearer economy principle. We have 
already seen that each interlocutor's best strategy choice depends on what 
the other party is doing. So, given oo and Po as a natural starting point we 
might want to define preferences simply in terms of expected utility: 


ULw) >(f,w) iff EUs(f’, Po, w) > EUs(J, Po, w) 
U,WI) > (f,w) iff EU Ac{w'z. 90, J) > EU R({w}, co, J) 


This simplifies to: 7 


(f',w) > (f,w) iff us(f', pol f), w) > us(f, po(J), w) 
U,WI) > (f,w) iff P(wil [f] > Pwl [f 


Observation 3.7. Interpretation based on optimal assertions PEAD) is 
strong Bi-OT's Q-principle: a form-meaning pair (f, w) satisfies the Q- 
principle iff o} (f) 40 and w E pPA(f). 


Proo]. A form-meaning pair (f, w) satisfies the Q principle iff there is no t' 
such that EU s(f’, Po, w) > EU 5 (J, Po, w) iff f is association optimal in w 
iffa (f) 40 and w E pPA(f). Q.E.D. 


Let's capture interpretation based on strong optimality in an interpre- 
tation operator for ease of comparison. If a {f) = O, the uninterpretabil- 
ity assumption holds, and we take pTI) = [fl]; otherwise: pet dd) = 
{w E W I(f,w) is strongly optimai}, which is equivalent to: 


pot (f) = {w € MaxvewP(ol [f]) | f € o1(w)}. (3.3) 


7 Originally, Blutner (1998) defined preferences in terrns of a function C that maps 
forrn-meaning pairs to real numbers, where C((J,w)) = c(J) x -log2 P(w] [JII). Form- 
meaning pairs were then ordered with respect to their C-value. Om formulation here 
amounts basically to the same, but further integrates the present assumption that 
casts are nominal and only sender relevant. 
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Proposition 3.8. For arbitrary interpretation frames it holds that p?7 (J) 
ie PONP: For interpretation frames satisfying strong expressibility it holds 


that PT (J) = pPA(J). 


Proo]. The first part is an immediate consequences of Observation 3.7. So 
assume strong expressibility and let a, (J) Æ 0 and w E pO4(J), so that 
f E o;(w). From Observation 3.4 we know that therefore w E po(J). So 
there is no wi’ for which P(wil [j]) > P(wl [f]]). But that means that (f, w) 
also satisfies the I-principle, and therefore w E p??(f). QED. 


Proposition 3.9. For arbitrary interpretation frames it holds that p97 (J) 
G PERON For interpretation frames satisfying strong expressibility it holds 
that pT (J) = pPÈ(J). 


Proo]. Let o} (f) 4 0 and w E p27 (f). Then w E MaxvEWP(vl [f]l) 
and f E øı(w). Suppose that there was a uit E W with P(w'|o;'(f)) > 
P(wloy*(f)). Then ui' E o} (f), but ui ¢ [FI This contradiets Observa- 
tion 3.1. The rest follows from Propositions 3.5 and 3.8. Q.E.D. 


3.4 Interpretation of optimal signals 

The results of the last sections are graphically represented in Figure 1. What 
do these results teil us about the respective interpretation rules? In par- 
ticular, what are the conceptual differences between the approaches? Can 
we conclude that one is better than the other? A quick glance at Equa- 
tions 3.1, 3.2 and 3.3 reveals that the only difference between frameworks 
lies in the treatment of probabilities.f The optimal assertions approach does 
not take probabilities into account, iterated best response chooses the most 
likely interpretations where the received message was optimal and Bi-OT 
chooses all those most likely interpretations given the semantic meaning of 
the message where that message was optima!. 

The simplest case where predictions differ is where the to be interpreted 
message f is true in three worlds, [f]l = {W, v, zz}, and optimal in two worlds, 
oj (f) = £v, u}, with varying degree of probability: P(w) > P(v) > P(u). 
In this case, the optimal assertions approach selects p?*(J) = øJ '(J) = 
{v, u}, iterated best response selects pB®(f) = {v}, while Bi-OT selects 
p?'(f) = 0. 

This seems to speak for iterated best response, maybe for optimal as- 
sertions, but somehow against Bi-OT. On the other hand, we might also 
credit Bi-OT for its strict continuation of the idea that probabilities eneode 
stereotypes in an associative salienee ordering: upon hearing f the associa- 
tions po(J) spring to mind and those are checked for optimality, so that, if 


8 Clearly then, for uniform probability distributions strong expressibility collapses into 
semantic expressibility and all frameworles behave the exact same. 


310 M. Franke 


OA ~RiOT 
`R 


FIGURE 1. Conneetion between (i) optimal assertions (OA), (ii) iter- 
ated best response (IER) and (iii) (strong) bidirectional optimality theory 
(BiOT): a straight arrow indicates inclusion of interpretations of signals 
while a dotted arrow with label SE indicates inclusion given strong express- 
ibility. 


the received message is not optimal in any ofthe associated worlds in po(J), 
then the receiver is stuck-at least for the time being; he might re-associate 
in a further step. 

Can we then make an empirical case for or against any candidate? A first 
observation is that all three approaches predict the l- and Q-implicature 
play equally wel!. In particular, since 1- and Q-frames satisfy strong ex- 
pressibility, the predictions for these cases are exactly the same for all three 
approaches. The M-frame, on the other hand, does not satisfy strong ex- 
pressibility, but nevertheless doesn't help judge frameworks, because all of 
the present candidates mispredict in this case. Take the M-frame as defined 
above. We then get: 


PPA = twv} pg) = fw, v) 
PP = {w} P(g) = [w,v} 
er (f) = {w} — p?*(g) = {wv} 


The problem is that none of the interpretation rules that we considered 
handles the long form 9 correctly. Can we fix this problem? 

The most obvious idea to try is further iteration. So what would the 
sender's best response g9 be to the receiver's strategy PI? The answer to this 
question now crucially depends on the uninterpretability Assumption 3.2. It 
is easy to verify that as long as v E PI(g), the sender's best response will be 
to send f in wand to send 9 in v. (Remember that costs are nomina!.) To 
this, in turn, the receiver's best response is the inverse of the sender strategy. 
The resulting play is indeed the M-implicature play. This is a noteworthy 
result in the light of the problem of equilibrium selection: iterated best 
response starting from a 'truth-only' sender strategy can account for 1-, Q- 
and M-implicatures for some versions of the uninterpretability assumption, 
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but not for others. (To wit, if Pl(g) = {w} iteration of best responses has 
reached a fixed-point different from the M-implicature play). 

So is the uninterpretability assumption in 3.2 defensible? It does not 
have to be, since at present it suffices to defend that Pl(g) ~ {w}, which 
implies that v E PI(g) as desired. And that Pl(g) Æ {w} can be argued for 
based on Cho's and Kreps' (1987) intuitive criterion, as has been demon- 
strated by van Rooij (2008) (see also the short discussion in Section 2.3). 
In simplified terms, the intuitive criterion gives a strong rationale why the 
receiver should not believe that a sender in w would send g: she has a 
message f that, given PI (J), is always better in w than signal 9 na matter- 
haw the receiver might react to g. (The signal 9 is equilibr-ium-daminated 
for w.) This reasoning establishes that w ¢ Pl(g), which gives us the M- 
implicature play immediately. If we adopt a weaker version and only require 
that Pl(g) A {w}, we can account for M-implicatures aftel' another round 
of iteration. 


4 Conclusion 


Taken together, we may say that, with only little help from the refinement 
literature, the present version of iterated best response provides a uniform, 
synchronie account of 1-, Q- and M-implicatures. It also subsumes, as a stan- 
dard game-theoretical model, the optimal assertions approach and strong 
Bi-OT. This does not discredit either of these latter approaches. For the 
optimal assertions approach is actually more general than presented here: 
its predictions were here only assessed for a special case, but the framework 
is not restricted to a sender who knows the actual world and a receiver who 
chooses interpretation actions. Similarly, strong optimality is not all there 
is to Bi-OT: there is also the notion of weak bidirectional optimality which 
also handles M-implicatures. The conneetion between weak optimality and 
iterated best response is not obvious and remains an interesting topic of 
future research. At present, we may safely conclude that, if game-theoretic 
standards are a criterion for our selection of models of implicature calcula- 
tion, then iterated best response fares best in the neo-Gricean terrain. 
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Introduction 


Markov decision processes model situations where a controller wishes to 
control optimally a system, taking her decisions in a sequential way and 
facing stochastic behaviour of the system. Step aftel' step, the Markov deci- 
sion process goes through a sequence of states sa, si, ... from a set of states 
S. At each step, the controller chooses an action a E A, which causes the 
process to change from state s to new state ¢ with fixed probability p(tls, a). 
The probability that the decision process stops is 0, i.e., \>,cg p(t|s,a) = 1 
and the time horizon is not bounded hence the decision process never stops. 
A history is an infinite sequence h = sOalsl... such that at each step 
n E N, the controller has chosen the action a,+/, knowing the sequence 
Sd, Sl, ..., Sn Of previous states. 

With each history h is associated a payoff y(h) E R given to the con- 
troller, and the goal of the controller is to maximize the expected value of 
her payoff. 

We are especially interested in these Markov decision processes where 
the controller can play optimally without having to memorize information 
about the history of the play: in this case the choice of an action a,+/ at 
step n by the controller only depends on the current state sn of the Markov 
decision process, the controller is said to play with a pure and stationary 
strategy. 

In this paper, we present a criterion about the payoff function which 
guarantees the existence of optimal strategies which are pure and stationary, 
in any Markov decision process with finitely many state and actions. 

Our result still holds in the broader framework of zero-surn perfect- 
information stochastic games, where the controller plays against an adver- 
sary which also chooses actions and tries to minimize the expected payoff. 
This was proven in (Gimbert, 2006a). However, we restriet here to the case 
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one-player games, i.e., Markov decision processes, since this framework is 
sufheient for giving some interesting applications of our result. 
Performances of reactive discrete-event systems can be measured in sev- 
eral ways and similarly Markov decision processes can be equipped with 
various payoff functions. Some well-known examples are the parity, the 
mean-payoff or the discounted payoff function, which were initially intro- 
duced in the broader context of zero-sum games. In Markov decision pro- 
cesses equipped with the discounted payofJ function, with each state s is 
associated a daily payoff r, and at each step the controller earns the daily 
payoff corresponding to the current state. Moreover there is an infiation 
phenomena: future payoffs are multiplied by a discount factor 0 < À < 1, 
and for a stream Ta, TI, ... of daily payoffs, the total payoff received by the 


controller is: 
`> A Ta . 


nen 


This payoff was introduced by Shapley (1953). Gilette (1957) considered 
the case where the controller seeks to maximize the average value of the 


stream of payoff, i.e., 


-  . f Ta +... +Tn 
hatiin i 


nEN n+1 
which defines the mean-payofJ function. 

Whereas discounted and mean-payoff games are used for economie mod- 
elling, paTity games were introduced for totally different purposes: they 
appeared in the context of theoretical computer science as a tooi for study- 
ing relations between a logic called the j-calculus and a class of computation 
models called tree automata (Emerson and Jutla, 1991; Gradel et al., 2002). 
The payoff computed by the parity payoff function depends on the set of 
states visited infinitely often. Other examples of payoff function are the 
limsup, liminf (Maitra and Sudderth, 1996) and the total (Thuijsman and 
Vrieze, 1987) payoff functions. 

Surprisingly, all these examples of payoff functions share a common non- 
trivial property. Indeed, in any Markov decision process equipped with one 
of these functions there exist optimal strategies of a very simple kind: they 
are at the same time pure and stationarij. A strategy is pure when the 
controller plays in a deterministic way and it is stationary when choices of 
the controller depend only on the current state, and not on the full past 
history. For the sake of concision, pure stationary strategies are called 
positional strategies, and we say that a payoff function itself is positional if 
in any Markov decision process equipped with this function, there exists an 
optimal strategy which is positional. 

Existence of positional optimal strategies has a strong algorithmic in- 
terest, since it makes the computation of optimal strategies easy. Indeed, 
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the class of positional strategies is finite, henee a naive algorithm eonsists 
in enumerating all the positional strategies and seleeting the strategy that 
gives the highest expeeted payoff. This algorithm is quite ineffieient in prae- 
tiee, sinee its running time is at least exponential in the size of the Markov 
deeision proeess, in the ease where expeeted payoffs are eomputable in poly- 
nomial time. However, for the diseounted, mean-payoffand parity funetions, 
the existenee of positional optimal strategies has been used to design poly- 
nomial time algorithms (Puterman, 1994; Filar and Vrieze, 1997). 


Content ofthe paper. This paper addresses the following question: what 
is the eommon property between the diseounted, mean, limsup and parity 
payoff whieh explains why all of them are positional? For answering this 
question, we introduee the class of submixing payoff funetions, and we prove 
that a payoff function whieh is submixing and prefix-independent is also po- 
sitional (ef. Theorem 2.3). This result partially solves our problem, sinee 
the parity, limsup and mean-payoff functions are prefix-independent and 
submixing (cf. Proposition 3.1). Our result has several interesting eonse- 
quenees. First, it unifies and shortens disparate proofs of positionality for 
the parity, limsup and mean payoff function (Seetion 3). Seeond, it al- 
lows us to generate a buneh of new examples of positional payoff functions 
(Seetion 4). 


Plan. This paper is organized as follows. In Seetion 1, we introduee notions 
of controllable Markov ehain, payoff funetion, Markov deeision proeess and 
optimal strategy. In Seetion 2, we state our main result: prefix-independent 
and submixing payoff funetions are positional (cf. Theorem 2.3). In the same 
seetion, we give elements of proof of Theorem 2.3. In Seetion 3, we show 
that our main result unifies various disparate proofs of positionality. In 
Seetion 4, we present new examples of positional payoff funetions. 


1 Markov decision processes 


Let S be a finite set. The set of finite (resp. infinite) sequenees on S is 
denoted S* (resp. SW). A probabilits; distriouiior: on S is a function 6:S > 
R sueh that Vs E S, 0 < d(s) < 1 and $9.6 0(s) = 1. The set of probability 
distributions on S is denoted D(S). 


1.1 Controllabie Markov chains and strategies 


Definition 1.1. A eontrollable Markov ehain A = (S, A, (A(S))sES'P) is 
eomposed of: 


e a finite set of states S and a finite set of aetions A, 
e for eaeh state s E S, a set A(s) CA of aetions available in s, 


e transition probabilities p :S x A — D(S). 
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When the current state of the chain is s, then the controller chooses an 
available action a E A(s), and the new state is t with probability p(tls,a). 

A triple (s,a,t) ES x A x S such that a E A(s) and p(tls,a) > 0 is 
called a transition. 

A historu in A is an infinite sequence h = Saalsl'" E S(AS)W such 
that for each n, (sn, an+l, Sn+l) is a transition. State Sa is called the souree 
of h. The set of histories with souree s is denoted P% ,. A finite historu 
in A is a finite prefix of a history. The set of finite histories (resp. of finite 
histories with souree S) is denoted P*, (resp. P% ,). 

A strategy in A is a function ø : P% — D(A) such that for any finite 
history h E P*, with target £ E S, the distribution o(h) puts non-zero 
probabilities only on actions that are available in t, i.e., (o(h)(a) > 0) => 
(ae A(t)). The set of strategies in A is denoted X 4. 

Certain types of strategies are of particular interest for us, these are pure 
and stationari] strategies. A strategy is pure when the controller plays in 
a deterministic way, i.e., without using any dice, and it is staiionars] when 
the controller plays without using any memory, i.e., her choices only depend 
on the cutrent state of the Markov decision process, and not on the entire 
history of the play. Formally: 


Definition 1.2. A strategy o E 3), is said to be: 
e pure ifVheEP*%, (a(h)(a) >0) => (o(h)(a) = 1), 
e staiionarij if Yh E P*, with target t, o(h) = a(t), 


e positional if it is pure and stationary. 


In the definition of a stationary strategy, t E S denotes both the target 
state of the finite history h E P*, and also the finite history £ E P% + of 
length 1. i 


1.2 Probability distribution induced by a strategy 
Suppose that the controller uses some strategy o and that transitions be- 
tween states occur according to the transition probabilities specified by 
p(|,.). Then intuitively the finite history Saal... anSn occurs with proba- 
bility 

o(so)(a@1) .p(sllsa, 41)+--o(So-* 8n—1)(@n) .P(SnISn-l, an). 


In fact, it is also possible to measure probabilities of sets of infinite 
histories. For this purpose, we equip P4, with a o-field and a probability 
measure. For any finite history h E Piy and action a, we define the sets 
of infinite plays with prefix h or ha: 


O, = {Saalsl'" E PY, 13n EN, Saal" Sn= h), 
Ona = {Saalsl'" E P4, 13n EN, Saal” Snan+l= ha). 
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Pas is equipped with the cr-field generated by the collection of sets ©, and 
Ona. A theorem of Ionescu Tulcea (cf. Bertsekas and Shreve, 1978) implies 
that there exists a unique probability measure PZ on PY s such that for any 
finite history h E P% s with target t, and for every a E A(t), 


Ps (Ona 10h) = o(h)(a), 
Pepe 10.. = p(Tlt, a). 


1.3 Payoff functions 

With each history is associated a real value, called the payoff and the con- 
troller seeks to maximize this payoff. Payoffs are computed by payofj" func- 
tions, in this subsection we give several examples of such functions: the 
mean-payoff, the discounted and the parity payoff function. 


1.3.1 Mean payoff 

The mean-payoff function has been introduced by Gilette (1957). It is used 
to evaluate average performances of a system. Each transition (s, a,t) is 
labelled with a daily payofj" T(S,a,t) € R. A history sOalsI' gives rise 
to a sequence TOTI ... of daily payoffs, where Tn. = T(Sn, an+], Sn+1)' The 
controller receives the following payoff: 


1 
CPmean(TOTI" ) = limsup — — Sori. (1.1) 
nEN n +1 i=0 


1.3.2 Discounted payoff 

The discounted payoff has been introduced by Shapley (1953) and is used 
to evaluate short-term performances. Each transition (s, a,t) is labelled not 
only with a daily payoff T(S, a,t) E R but also with a discount factor 0 < 
A(s,a,t) < 1. The payoff associated with a sequence (Ta, ào)(r1, A1) E 
(R x [0,1)W of daily payoffs and discount factors is: 


Paise( (To, Xo)(71,A1) +++) = ro +Aori + Aoro + °° (1.2) 


The discounted payoff has an intuitive interpretation. Suppose that each 
time a transition (s, a,t) occurs, a biased coin is tossed to know if the system 
may halt or proceed. The system proceeds with probability A(s,a, t), and 
halts with probability 1- A(s, a, t). Ifthe system halts at step n then the 
payoffis the sum TO+" .+Tpn ofrewards seen so faro With that interpretation, 
Pise is exactly the expected sum of daily payoffs before the system halts. 


1.3.3 Limsup and liminf payoff 


The limsup and liminf payoff functions can be used to measure the peak 
performances of a system. Let C C R be a finite set of real numbers, and 
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coc1... ECW. Then 


Ylsup(Cocz. ..) = lim sup cn 


n 


Plint(coci ...) = lim inf cn- 
n 


1.3.4 Parity payoff 

The parity payoff function is used to encode temporal logic properties 
(Grädel et al., 2002). Each transition (s,a,t) is labelled with some pri- 
ority c(s, a,t) E N. The controller receives payoff 1 if the highest priority 
seen infinitely often is odd, and 0 otherwise. For coc1 ... E NY, 


phe (a K ) {0 if lim sup., cn is even, (1.3) 


1 if lim sup., cn 1s odd. 


Remark that since we only consider controllabie Markov chains with finitely 
many States, lim sUPnc, is always finite, hence the parity payoff function is 
well-defined. 


1.3.5 General payoffs 

In the sequel, we will give other examples of payoff functions. Observe that 
in the examples we gave above, the transitions were labelled with various 
kinds of data: real numbers for the mean-payoff, couple of real numbers for 
the discounted payoff and integers for the parity payoff. 

We wish to treat those examples in a unified framework. For this reason, 
we consider in general that a controllabie Markov chain A comes together 
with a finite set of colours C and a mapping col: Sx A x S — C, which 
colours transitions. 

Inthe case of the mean payoff, transitions are coloured with real numbers 
hence C C R, whereas in the case of the discounted payoff colours are 
couples C < R x [0,1[ and for the parity game colours are integers C < N. 

For a history (resp. a finite history) h = soats1..., the colour of the 
history h is the infinite (resp. finite) sequence of colours 


col(h) = col(so, al, S1) COKSI, a2, S2)--- 


Aftel' a history h, the controller receives payoff cp(col(h)), where pay is 
a payoff function which associates a payoff with each infinite sequence of 
colours: 


Definition 1.3. Let C be a finite set. A payoff function on C is a function 
y : CW — R which is bounded and rneasurable for Lhe cr-field generaled by 
the sets {uC”, u. E C*}. 


Boundedness and measurability of payoff functions guarantee that the 
expected payoff is well-defined. 
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1.4 Values and optimal strategies in Markov decision processes 


Definition 1.4. A Markov decision process is a couple (A, p), where A is 
a controllabie Markov chain coloured by a set C and y is a payoff function 


on C. 


Let us fix a Markov decision process M = (A, p). Aftel' history h, the 
controller receives payoff co(col(h)) E R. We extend the definition domain 
of y to P% ,: 

Yh E Ps: g(h) = ep(col(h)). 


The expected value of y under the probability PJ is called the expected 
payofj" of the controller and is denoted E?[y]. It is well-defined because ọ is 
measurable and bounded. The value ofa state s is the maximal expected 
payoff that the controller can get: 


val(M)(s) = sup EŞ [|y]. 
cela 


A strategy ø is said to be optimal in M if for any state s E S, 
E; [p] = val(M)(s). 


2 Optimal positional control 


We are interested in those payoff functions that ensure the existence of 
positional optimal strategies. These are defined formally as follows. 


Definition 2.1. Let C be a finite set of colours. A payoff function y on CW 
is said to be positional if for any controllabie Markov chain A coloured by 
C, there exists a positional optimal strategy in the Markov decision process 


(A, p). 


Our main result concerns the class of payoff functions with the following 
propetties. 


Definition 2.2. A payoff function y on CW is prefix-irulependeni if for 
any finite word u. E C* and infinite word v E CW, pluv) = (v). Pay- 
off function y is submixing if for any sequence of finite non-empty words 
UO, VO, U1, U1, --. E C*, 


p(uovowsvi---) < max {yp(uows---), p(vowr--+)}- 


The notion of prefix-independence is classica!. The submixing property 
is close to the notions of fairly-mizing payoff functions introduced in (Gim- 
bert and Zielonka, 2004) and of concave winning conditions introduced in 
(Kopczyriski, 2006). We are now ready to state our main result. 
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Theorem 2.3. Any prefix-independent and submixing payoff function is 
posi tional. 


The proof of this theorem is based on the 0-1 law and an induction on 
the number of actions and can be found in (Gimbert, 2006b). We do not 
repeat this proof here as we prefer to present some of its applications, in 
the next two sections. 


3 Unification of classical results 


Thanks to Theorem 2.3, it is possible to give a unified proof of position- 
ality for the limsup, the liminf, the parity and the mean-payoff function. 
Indeed the positionality of these payoff functions is a simple coroliary of the 
foliowing proposition. 


Proposition 3.1. The payoff functions Yjsup, lint) Ppar and Ymean are 
submixing. 


Proo]. Consider a finite set of real numbers C C R and a sequence of 
finite non-empty words wo, va, w1, V1, E C*. Let uw = wouy--: E CW, 
v = vow... E CW and w = ugvouivı E C'". The foliowing elementary 
fact immediately implies that Yjsup is submixing: 


Pisup(w) max {Yisup(w), Pisup(v)}. (3.1) 


In a similar way, Yjin¢ is submixing since 


Piint(w) min{ Yiine(2), Yiine(v) }. (3.2) 


Now suppose that C = {a,...,d} happens to be a finite set of integers 
and consider function par. Remember that CPpar(w) equals 1 if cplsupcw) is 
odd and 0 if cplsup(w) is even. Then using (3.1) we get that if CPpar(w) has 
value 1 then it is the case of either Ppar(t) or CPpar(v). It proves that Ypar 
is also submixing. 

Finally, we show that function Ymean is submixing. Again C C Ris a 
finite set of real numbers. For i E N let ci E C be the ith letter of word 
w. Since word w is a shuffle of words u. and v, there exists a partition 
(Ia, I1) of N such that u. = (Cciiefo and v = (Cdien. For any n E N, let 
Ig = Jan {a,...,n}/ and J? = I,n {a,...,n}. Then forn EN, 


il < Wel f 1 UT 1 
n4 Toe n+1\ Hl a" "n+l M? 2 "4 


i=0 i£ 
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The inequality holds since Ue + m = 1. Taking the superior limit of this 
inequality, we obtain CPmean(w) < Max{Pmean(U), Ymean(v)}. It proves that 
Pmean is Submixing. Q.E.D. 


Since Plsup; Plinf, Ypar and Ymean are clearly prefix-independent, Propo- 
sition 3.1 and Theorem 2.3 imply that those four payoff functions are posi- 
tional. 

This technique gives a uniform proof of several disparate results, and we 
compare it to existing proofs. 

The case of Markov decision processes equipped with a parity criterion 
was treated in (Courcoubetis and Yannakakis, 1990). The proof is by in- 
speetion of strongly connected components whose maximal priority is odd. 
This proof is far more simple than the result of Gimbert (2006b) used in 
our argument. 

The case of limsup and liminf Markov decision processes was treated 
in (Maitra and Sudderth, 1996) in the broader framework of stochastie 
games with infinitely many states. In (Maitra and Sudderth, 1996) values 
Markov decision processes equipped with sup and limsup payoff functions 
are characterized as fixpoints of some operators. The existence of pure and 
stationary optimal strategies in the case of finitely many states is derived 
from this characterization. The proof of Maitra and Sudderth and the proof 
given here use radically different techniques. The proof of Gimbert (2006b) 
is shorter since we do not rely on a fine study of values of limsup and liminf 
games. 

For mean-payoff Markov decision process, there basically exists two 
proofs of the existence of pure and stationary optimal strategies. The first 
approach, that can be found for example in (Neyman, 2003), consists in 
proving existence of such strategies in discounted Markov decision processes, 
and using the fact that values of discounted Markov decision processes are a 
rational function of discount factors. This implies existence of pure station- 
ary strategies that are optimal for every small values of discount factors, a 
phenomenon called Blackwell optimality. In particular, pure and stationary 
strategies that are Blackwell optimal are optimal in the limit mean-payoff 
Markov decision process. 

Another approach, in two steps, consists in first considering a weak form 
of mean-payoff Markov decision processes, where payoffs are computed tak- 
ing the average value of expectations of rewards rather than the expectation 
of the average value of rewards (see Puterman, 1994 for example). Using 
simple matrix calculation, it can be shown that for this weak form of mean- 
payoff Markov decision processes, there exists pure and stationary optimal 
strategies. Then one can conclude using a non-trivial result of Bierth (1987) 
that these strategies are also optimal for (not weak) mean-payoff Markov 
decision processes. 
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In our case, we directly prove the existence of pure and stationary op- 
timal strategies, under very weak hypothesis that include as special cases 
the classes of limsup, liminf, mean-payoff and parity Markov decision pro- 
cesses. The proof of Gimbert (2006b) is more elementary than the proofs 
above, since it uses only elementary probabilities and measure theory. The 
theorem of Gimbert (2006b) is also more powerful: in the next section we 
present several examples of payoff functions that can be proven to be po- 
sitional thanks to our result, but we do not know if it is possible to do so 
using existing techniques of Maitra and Sudderth (1996), Puterman (1994), 
Bierth (1987), and Neyman (2003). 


4 New examples of positional payoff functions 


In this section we present two new examples of positional payoff functions, 
namely the weighted payoff and the compromise payoff. We also present 
three operations on payoff functions, namely mixing with the liminf payoff 
function, approximation and hierarchical product. These operations have a 
nice property: the class of submixing and prefix-independent payoff func- 
tions is stabie under these operations, hence these operations can be used 
to generate numerous new examples of positional payoff functions. 


4.1 The weighted payoff 


Weighted payoff functions were recently introduced by Gimbert and Zielonka 
(2007). Each transition is labelled with a couple of rewards and weights, 
the last being a strictly positive real number: the set of colours is C C 
{(T,W) Ir E R,w E R,w > O}. A history sOalsl--- gives rise to a sequence 


(Ta, WO)(TI, w1): .. of rewards and weights and the controller receives the 
payoff: 
1 n 
cPweight((To, WO)(TI, W1)**.) = limsup an W Swi Ti: (4.1) 
neN i=O 7% i=0 


This function generalizes the mean-payoff function: if all weights are 
taken to be 1, then values of the mean-payoff function and the weighted 
payoff function do coincide. 

Intuitively, weights used in the definition of the weighted payoff functions 
can be considered as time lengths, and rewards as instant performances. 
With this interpretation, CPweight computes the average performances over 
time. 

The weighted payoff function is positional. Indeed, it is sub-mixing, the 
proof is very similar to the proof of Proposition 3.1. Moreover, CPweight is 
also clearly prefix-independent, hence Theorem 2.3 implies that CPweight is 
posi tional. 
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Weighted Markov decision processes have strong connections with dis- 
counted Markov decision processes with multiple discount factors (Gimbert 
and Zielonka, 2007), that extend the well-known connections between dis- 
counted and mean-payoff Markov decision processes (Gilette, 1957). 


4.2 The compromise payoff function 


The compromise function was introduced in (Gimbert and Zielonka, 2004), 
and is defined as follows. We fix a factor À E [0,1], a finite set C C R and 
for u. E C", we define 


Cover(s) A: Pisup(%) ł (1 = A) ' Piint (2). (4.2) 


This function generalizes the limsup and liminf payoff functions, which 
correspond to the case where A = O or à= I. 

Intuitively, peak performances of a system can be evaluated using the 
limsup payoff, whereas its worst performances are computed using the liminf 
payoff. The compmmise payofj" function is used when the controller wants 
to achieve a trade-off between good peak performances and not too bad 
worst performances. 


4.3 Mixing with the liminf payoff 

Not only the limsup function Yjsup but any payoff function p may be mixed 
with the liminf function in a way similar. The nice property of this op- 
eration is that the submixing property is preserved, as stated in the next 
proposition. 


Proposition 4.1. Let C < R, 0 < A < 1 and y be a payoff function on 
C. Suppose that ọ is prefix-independent and submixing. Then the payoff 
function 


A.p+(1- A). Giint (4.3) 
is also prefix-independent and submixing. 
Proo]. Let C CR be a finite set of real numbers and wo, va, u1, V1, E C* 
be a sequence of finite non-empty words over C. Let u. = wor, E CW, 
v = vow ... E CW and w = wovot vi ... E CW. Then since ọ is submixing, 
p(w) < max{y(w),y(v)} and moreover cpiinf(w) = min{ Piine(e), CPlinfcv)}, 
This proves that A. + (1 - A). Ying is submixing. Q.E.D. 


In particular, when in (4.3), ¢ is either Ymean; Ppar OF Yisup, We obtain 
several new examples of positional payoff function. 
4.4 Approximating a payoff function 


Approximation of a payoff function y : CW — R consists in composing y 
with a non-decreasing function f :R — R. For example, if f is the threshold 
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function 159, which associates 0 with strictly negative real numbers and I 
with positive number, then foy indicates whether ọ has positive or negative 
value. 

If p is positional then of course fo y also is, since a positional optimal 
strategy for some Markov decision process (A, y) will be optimal for the 
Markov decision process (A, f o p) as weil. In fact, it is straightforward 
to check that approximation not only preserves positionality but also the 
submixing property. 


4.5 The hierarchical product 
Now we define a binary operator between payoff functions, which stabilizes 
the farnily of prefix-independent and submixing payoff functions. We call 
this operator the hnerarchical product. 

Let po, y, be two payoff functions on sets of colours Co and Cl respec- 
tively. We do not require Co and Cl to be identical nor disjoints. 

The hierarchical product Yo I> %1 of Yo and y; is a payoff function on the 
set of colours COUC] and is defined as follows. Let u.= COC]... E (CoUC,)” 
and wo and w, the two projections of u.on Co and Cl respectively. Then 


polu if uo is infinite, 
(po b pi)(u) = CM? | 
yilu,) otherwise. 


This definition makes sense: although each word wo and u; can be either 
finite or infinite, at least one of them must be infinite. 

The parity payoff function has an alternative definition in term of the 
hierarchical product. For e € N, let Oe and Ie be the payoff functions defined 
on the one-Ietter alphabet fe} and constant equal to 0 and I respectively. 
Let dbe an odd number, and Ypar be the parity payoff function on {0, ... , a}. 
Then 

Ppar = Id l> Od-l l> ... l> lı I> 00. 


The submixing property is stabie under the hierarchical product: 


Proposition 4.2. Let Yo and y be two payoff functions. If Yọ and %1 are 
prefix-independent and subrnixing, then yo l> Y; also is. 


Hierarchical products of weighted payoff functions are tightly linked 
with the discounted payoff function. Indeed, in a Markov decision pro- 
cess equipped with a discounted payoff function, when the discount factors 
converge to 0, then under weak hypothesis the values converge to the values 
of the same controllabie Markov chain equipped with a hierarchical product 
of weighted payoff functions (Gimbert and Zielonka, 2007). 
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5 Conclusion 


We introduced the class of prefix-independent and submixing payoff func- 
tions and proved that this class enjoys a nice property: in any Markov 
decision process equipped with one of these payoff functions, there exists 
optimal strategies that are pure and stationary. Moreover this class is robust 
since it contains several payoff functions that are central tools in economics 
(mean-payoff and limsup functions) and computer science (parity function). 
Based on these results, we were able to exhibit several new examples of po- 
sitional payoff functions. 

The results of the last section give rise to natural algorithmic questions. 
For Markov decision processes equipped with mean, limsup, liminf, parityor 
discounted payoff functions, the existence of optimal positional strategies is 
the key for designing algorithms that compute values and optimal strategies 
in polynomial time (Filar and Vrieze, 1997). For examples generated with 
the mixing operator and the hierarchical product, it seems that values and 
optimal strategies are computable in exponential time, but we do not know 
the exact complexity. Also it is not clear how to obtain efficient algorithms 
when payoff functions are defined using approximation operators. 

Another interesting direction for future work is the definition of a good 
quantitative specification language for optimal controller synthesis of reac- 
tive pTogmms, i.e., programs interacting with their environment. Temporal 
logies such as CTL* may be used for specifying logical properties about the 
behaviour of reactive programs, indeed CTL* enjoys nice properties such 
as closure under boolean operations and computational tractability. When 
the environment is modelled by stochastic and non-deterministic transitions, 
computing the minimal probability for a reactive program to satisfy a CTL* 
specification amounts to solving a Markov decision process equipped with 
a parity payoff function. The designer of a program may be less interested 
in the correctmess of the program than in its performances, for example the 
expected RAM use. There does not exists a language for specifying quan- 
titative properties of programs with the nice properties of CTL*, although 
there have already been several steps in this direction (Baier and Clarke, 
1998; McIver and Morgan, 2002; de Alfaro, 2003; Chatterejee et al., 2004; 
de Alfaro et al., 2004; Lluch-Lafuente and Montanari, 2005). Results of the 
present paper is another effort towards the definition of such a quantitative 
specification language. 

To conclude, we formulate the following conjecture about positional pay- 
off functions: "Any payoff function which is prefix-independent and posi- 
tional for the class of non-stochastic one-player games is positional for the 
class of Markov decision processes" . 
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